ECO Flow:
-----------
Many types of ECO flow supported in Encounter L/XL/FE-L/FE-XL (Enc version 7 and above). GXL and conformal ECO have additional support for ECO. GXL supports ecoRemap, while conformal supports conformalECO.

These ECO flows supported in Enc:

1. Pre-Mask changes from ECO file => note: eco file generates a new verilog netlist, which is used by subsequent steps.
2. Pre-Mask changes from new verilog netlist =>
loadConfig old.conf, set rda_Input(ui_netlist) "newchip.v", ecoDefIn oldchip.def, ecoPlace, ecoRoute, ..
3. Pre-Mask changes from new DEF file => similar as 2 above, as def file contains new logical cells/connections. An ECO file is generated by ecoCompareNetlist, loadECO loads this eco file, and then follow as in 2.
4. Post-Mask changes from new verilog netlist => use -postMask for ecoDefIn to minimize mask changes. Otherwise same as pre-mask.
5. Post-Mask changes using ECO spare cells or GA(gate array) cells. => preferred method at TI

-----------------------------------
2 methods for generating new eco netlist: (we've the new 1p1 rtl with our fixes in)
1. Manual: Here, we look at 1p0 netlist in debussy and figure out where to place gates in 1p1 netlist. Then, we an modify netlist in 2 ways: (In both of these ways, new netlist only has new gates added with appr connections. These new gates are not connected or matched to spare gates as the VDIO tool is supposed to do those connections).
 A. eco directive file: create eco_directive file that has the list of gate changes and connections. What this does is that it adds new gates with appr connections to netlist. We should look at this new netlist generated before we move to VDIO. Then we read in this file in VDIO.
 B. directly modify old ntlist: We directly modify old netlist and then read in the new netlist. This new netlist has new gates added with appr connections. It's similar to above option except that we don't use eco directives.

Then we run VDIO which reads in the old def and the new netlist (either eco directive or new verilog netlist). The tool then matches the changes with spare gates, places them and routes it to create lec clean netlist and def. then we run final checks, timing, etc. We discuss this methos below.

2. Conformal: Here we use conformal lec. We modify rtl for 1p1. Then we run thru Synthesis to create 1p1 netlist. Then we run conformal lec which diffs old 1p0 PnR netlist with newly synthesized 1p1 netlist. It creates a patch, and then generates a new netlist with the changes. This new netlist uses spare cells, so spare cells are already mapped to new logic within netlist (you'll see that spare cells are removed from the spare module). In manual option above, no spare cells were mapped to new logic. Then we go thru regulat PnR flow to accomodate this patch in VDIO. We dsicuss this method in later section marked conformal_eco.

NOTE: Regardless of which method we use for generating netlist, we have to run ecoDesign in VDIO (super cmd which does everything for us). We can also make changes directly to verilog and run ecoDesign.
#ecoDesign cmd is supported in all Enc version, so use ecoDesign. It takes EDI System database and a modified netlist as input and performs ECO operations. It restores the design, examines the changes in the new netlist, and automatically implements the required  changes  with ecoPlace and ecoRoute. deffile is not there in enc.dat database, but it's OK as rout.gz and place.gz has that info.
ecoDesign -postMask -modifyOnlyLayers <MLb>:<MLt> -spareCells <spareCellName> -useGACells <GACoresite> <old_design.enc.dat top_cell new_netlist> => Use -noEcoPlace -noEcoRoute if we don't want to ecoplace and ecoroute with this cmd, but want to do it separately.
 
---------------------------------------
Manual ECO (non conformal):
---------------------------------------
Inreactive ECO: provides manual inc updates to design.
#we can also do an interactive ECO by going to optimize->interactive ECO for PreMask changes. We can add repeater(ecoAddRepeater), upsize/downsize instances(ecoChangeCell), delete buffers(ecoDeleteRepeater), display buffer tree to modify it(displayBufTree)

#For PreMask/PostMask ECO, we can also do file->ECO design. Then goto Place->ECO Place for placement. then do routing by route->Nanoroute (choose ECO route) => instead of doing it on the encounter cmd line

------------------------------------
ECO changes using ECO spare cells (post mask):
------------------------------------
Flow for Making changes from ECO file: (preferred method at TI)
load old config file => load new_change.eco file (modifies old verilog to get new verilog) => ecoDefIn old_def file => specifySpareGate => ecoPlace => ecoRoute =>save design

#0. make new(1p1) dir and cp files from old(1p0) dir & then change mode:
cp -rf /db/YELLOWSTONE/design1p0 /db/YELLOWSTONE/design1p1
chmod 777 -R /db/YELLOWSTONE/design1p1

1. Update RTL in Source dir.
#run debussy in old dir on gate level netlist to look at gates to make changes in gate level netlist
cd /db/EPSTINGRAY/design2p0/HDL/Debussy.
Run create_symbols if *.lib++ dir not present
run_debussy => runs debussy with/without any options. If with options, "-f <gate_netlist>". If w/o options, bring up debussy, click  File->Import design, Put the file name (/db/DRV9401/design1p0/HDL/FinalFiles/digtop_VDIO.v) in bottom box, and then click Add, then OK.

2A. Run ECO flow in VDIO after running encounter (tcl/eco_flow.tcl => this file has all cmds in it for eco flow)
NOTE: to add/delete any pins, use PinEditor in VDIO gui (Edit->PinEditor). Do this before running ecoplace. Else, new pins are added at origin (0,0). To edit/add pin, you can also use "editPin" cmd:
#editPin -pinWidth 0.4 -pinDepth 0.4 -fixOverlap 1 -side Left -layer 3 -assign 0.0 367.85 -pin RX_SEL[4]

1st option: run ecoDesign with no place and route. This super cmd is explained above.
-----------
ecoDesign -postMask -noEcoPlace -noEcoRoute -spareCells spr_*/spr* dbs/filler/filler.enc.dat digtop /db/HAMMER_OA/design1p1/.../digtop_final_route_eco.v

2nd option: do it old way of reading in 1p0 config, 1p0 def and 1p1 directives.
----------
loadConfig dbs/filler/filler.enc.dat/digtop.conf => load previous config file for VDIO. Change dir path (set cwd) to present dir in this file. Config file only has path locations of digtop.v (note that even though this is verilog file for filler, it doesn't have any filler cells in it, as they don't exist in tech .lib file), tech .lib and tech .lef files. It doesn't have def file info, so we have to do DefIn to read def file.

# Read 1p1 eco_directives => This adds new cells, makes new connections etc to 1p0 verilog file to make new 1p1 file.
source tcl/eco_directive_1p1.tcl
# Or other option is to modify old netlist manually to create new netlist, and read that new netlist.
#set rda_Input(ui_netlist) "/db/.../digtop_final_route_1p1.v" => This overwrites the "i/p netlist" in loadConfig above
#commitConfig => This commits the config file so that all parameters spec above are applied


# ECO DEF in the old DEF file. since new verilog design is in memory and we are reading old def file, tool can figure out new changes for 1p1.
#-useGAcells GACoresite => specifies GA Core site to use for gate array eco. In cell lef file, it looks for cells with "SITE = GACoresite name" specified here. Regular stdcells have "SITE CORESITE", while GA cells have "SITE GACORESITE". That's how tool knows which cells are ECO cells that can be built from filler cells. this cmd implies "postmask" mode.
#-suffix _SPARE_DELETED => Appends the specified suffix to cells that appear in the DEF file but have been deleted in the new netlist. Default: _SPARE
#-postMask => When used with -postmask option, tool can only change nets, not cells. tool checks for cells that exist in memory but not in def, and marks them as unplaced. It then maps these to fillers/spares during ecoPlace. Modified nets which are found in both memory and DEF file, but whose connections are different, are processed during ecoRoute. When -postMask option is not used, it implies pre-mask mode, which can change cells too (it can put any new cell in empty space of fillers).

ecoDefIn -postMask -reportFile ecoDefIn.rpt /db/YELLOWSTONE/design1p0/HDL/FinalFiles/digtop/digtop_final_route.def => Restores physical information from an old design and compares this information with the new design (modified verilog using directives bove). It gives report on screen saying what new inst/net were added, etc (also dumps it in report file ecoDefIn.rpt specified above).

# Specify Spare Gate (-inst specifies instance name and NOT module name) This is needed only if we are not using gate array as spares (i.e. GA cells are not specified above).
specifySpareGate -inst spr_*/*spr_* => use any spare cell in spare modules.
specifySpareGate -inst spare_*/spare_inv_* => if you want to use only inverters
specifySpareGate -inst I_scan_iso_out/g1453 => This specs any extra gate (unused) as a spare cell. This is useful when we have some usuned gates in netlist that we want to use for eco purpose.

After running one of the options above, run ecoPlace and ecoRoute.
-------------
ecoPlace -useSpareCells true

#user intervention to change spare cell mapping. Provide instance name and NOT module name
ecoSwapSpareCell i_inst/eco_inst_an2 spr_3/spr_gate65 => gate "spr3/spr_gate65" from spare cell module is mapped to i_inst/eco_inst_an2. Here eco instance an2 was already mapped to some spare cell, but we didn't like the mapping, so we swap this cell with this other spare cell. Now, spr_gate65 becomes eco_inst_an2, and eco_inst_an2 becomes spr65

# ECO Route new netlist. If only certain metal layers, specify them
ecoRoute -modifyOnlyLayers 1:3
setNanoRouteMode -quiet -drouteFixAntenna true => set this if antenna errors still remain
ecoRoute -modifyOnlyLayers 1:3 => rerun eco route if errors are still there

#ecoRoute may not be able to route because it doesn't touch non-eco nets. Rerun ecoRoute until all errors are fixed. If errors still remain, we can run Nanoroute directly in eco mode. However, ecoRoute is still preferred to be run since it does preprocessing which minimizes routing changes. Cmds below do the same job as ecoRoute above, but can move non-eco nets too.
setNanoRouteMode -quiet -drouteFixAntenna false => optional, improves routing.
setNanoRouteMode -quiet -routeWithEco true
setNanoRouteMode -quiet
#setNanoRouteMode -routeEcoOnlyInLayers 1:3 => can use this single cmd or use these 2 cmds:
setNanoRouteMode -quiet -routeBottomRoutingLayer 1 => bottom routing layer has to be the lowest layer on which there are existing nets. Otherwise error says: "conflict with already existing routed wires on layer x-1"
setNanoRouteMode -quiet -routeTopRoutingLayer 3 =>  similarly, top routing layer has to be the highest layer on which there are existing nets.
setNaonoRouteMode -quiet routeSelectdNetOnly false
routeDesign -globalDetail => instead of this, we can also use: globalDetailRoute 100.0 1200.0 350.0 600.0 => specify co-ords if you want to reroute within a certain area. Although globalDetailRoute and "routeDesign -globalDetail" seem to be doing the same thing, they result in different results. "routeDesign -globalDetail" gives better results

#NOTE: keep on rerunning "routeDesign -globalDetail" until it passes all drc. (set antenna fix to false)
#Do not use the globalRoute command in ECO mode (use globalDetailRoute as shown above. globalRoute only performs global routing, while detailRoute only performs detailed routing).
#If more than 10 percent of the nets are new or partially routed, run full global and detailed routing instead of ECO routing (set routeWithEco false so that routing is done from scratch. Most of the times, it fixes all routing issues.)

#to route only a list of nets, which are in selectnets.txt file (one net per line)
set NET_FILE [open "selectnets.txt"]
foreach i [ read $NET_FILE ] {
 selectNet  $i => can only select one net at atime. wildcards are allowed
}
close $NET_FILE
#route these nets
setNanoRouteMode -routeSelectedNetOnly true => routes selected nets only. default is false (routes all nets).
routeDesign -globalDetail

#set_attribute -net <netName> -skip_routing true => to skip routing on selected nets, useful when we don't want to touch nets which are on layers above or below the eco routing layers. Since we can only specify one net_name, we have to use this cmd multiple times for multiple nets. However, this is dangerous to use, as set_attribute ties the attribute with that net, and it is saved with the database. so, next time encounter runs on this database, this attribute is still there, unless we set attribute to false.

# Save design
saveDesign ./dbs/eco_filler_1p1/eco_filler_1p1.enc
checkDesign -noHtml -all -outfile ./dbs/eco_filler_1p1/eco_check_design.rpt => checks design for all issues. Necessary as final_check.tcl later doesn't check for floating nets, etc.

2B. with eco_flow.tcl, we saved the new db into filler.enc. So, we need to run steps beyond filler, to run all checks and get the final netlist.
#Note: we should run timing as extra step since we should make sure design is timing clean, before we run PT.
timeDesign -signoff -reportOnly       -prefix digtop_post_route_signoff
timeDesign -signoff -reportOnly -hold -prefix digtop_post_route_signoff

#run post route opt if timing not met or any other violations
setOptMode -effort high
optDesign -postRoute -hold -prefix digtop_post_route_opt

#now run steps beyond filler
source tcl/final_check.tcl => to verify conn, etc.
source tcl/export_final.tcl => run extractRC to generate spef, get final verilog and defout.

3. run Formality to check RTL against ECO netlist. (For ECO netlist, use verilog generated above)
4. Rerun PT to check if timing is ok
5. Rerun all RTL sims and gate sims
6. Regenerate Tetramax patterns and rerun gatesims
7. Import Netlist and DEF to Cadence for top sims and tapeout

------------------------------------
ECO changes using gate array cells (post mask):
------------------------------------
When we use GA cells, we don't have any GA cells in netlist. For doing ECO change, we modify the original netlist to add ECO cells ending in E (made from GA cells) , and then in the layout, appr filler cells are connected to form these GA cells.
These ECO cells have a prefix E at the end to indicate that they are GA cells (i.e IV110E). These cells are generated from base filler cells (FILLER5, FILLER10, etc) which are present in layout. These filler cells were inserted in layout during the filler step (where these base filler cells are filled first and then any remaining spaces are filled with Dcap fillers). Tool figures which E cells can be replaced with which FILLER cells is by looking at physical of cells. NOTE: filler cells are never removed from design. Tool just picks up appr space where ECO cell can be placed, and makes metal connections to reflect the change. Filler cells are still unmodified under the ECO cell.

1. first make sure that ECO filler cells were put in 1p0 design using these 2 cmds:
addFiller -cell  FILLER5 FILLER10 FILLER15 FILLER20 FILLER25 FILLER30 FILLER40 FILLER50 FILLER55 -prefix FILLER_ECO
addFiller -cell  SPAREFILL1 SPAREFILL2 SPAREFILL4 SPAREMOSCAP3 SPAREMOSCAP4 SPAREMOSCAP8 -prefix FILLER_NORMAL

2. get the i/p netlist from 2p0 and modify it manually to add the new eco cells (ie IV110E, AN210E, etc) that need to be added, connecting them appropriately. name it as dig_top_noPhys_2p1.v
ex:    IV120E eco2_2p1_inv27 (.Y(eco2_2p1_prdata_27_bar), .A(eco2_2p1_prdata_27));

3. Once done with changes, read the old layout db and new netlist
ecoDesign -postMask -noEcoRoute -noEcoPlace dbs/handoff_20121106.enc.dat dig_top /data/PROJECT/.../dig_top_noPhys_2p1.v => we don't do eoPlace and ecoRoute as we do it in separate steps

4. do ecoplace using GA cells
ecoPlace -useGAFillerCells "FILLER55 FILLER50 FILLER40 FILLER30 FILLER25 FILLER20 FILLER15 FILLER10 FILLER5" => these filler cells should not have FIXED attribute in def file, else tool would not pick these for replacement, and will try to move around other std cells (which is incorrect).

5. If placement causes any errors (like overlapping placement etc), fix it by deleting and moving
eg: deleteInst FILLER_pdLogic_18457 (delete and then move/add filler cells manually and place them at correct location)

6, do ecoRoute
ecoRoute

7. If ecoRoute doesn't fix all routing violations even after multiple attempts, do full blown routing. Can be done from GUI also: Route->NanoRoute->Route. Unselect ECO ROute and select "Global Route, Detail Route".  Check "Fix Antenna". If you want to fix net by net, select "selected nets only" and select on those nets on encounter layout gui.
   setNanoRouteMode -quiet -routeWithEco true => may be set to false if we want to do full blown route
   setNanoRouteMode -quiet -drouteFixAntenna true
   setNanoRouteMode -quiet -routeTopRoutingLayer default
   setNanoRouteMode -quiet -routeBottomRoutingLayer default
   setNanoRouteMode -quiet -drouteEndIteration default
   setNanoRouteMode -quiet -routeWithTimingDriven false
   setNanoRouteMode -quiet -routeWithSiDriven false
   routeDesign -globalDetail -viaOpt -wireOpt

8. confirm location of newly added cells and then save new db
 selectInst *_2p1*
 saveDesign ./dbs/handoff_eco_2p1

9. Now run steps 2B and beyond (step 2B, 3-7) as shown in Normal filler cell flow. Run timing, optDesign and then steps beyond filler.
----------------------
ECO directives (old way):
---------------------
to make new connections, we use the 5 cmds listed below in tcl/eco_directive_1p1.tcl.
NOTE: all cmds specify instance name of any module, and NOT the module name itself. However, if we use -moduleBased, then we specify module_name itself. As module_name is unique for each instance in the synthesized netlist (see final synthesized netlist format desc above), it works OK.

addModulePort: To add port or bussed port to a module. Module should not have net with that name. Ex:
addModulePort i1/i2/i3 p1 input => adds i/p port p1 on instance i3(in hier i1/i2).

attachModulePort: Attaches a port in the specified instance (or top level) to a net. Seems like this cmd attaches ports to nets outside the module (i.e the net has to be at a higher level of hier than the port). This cmd doesn't detach anything, so detachModulePort cmd also needed. (this is different than attachTerm which does detach automatically)
attachModulePort i1/i2/i3 p1 i1/i2/n1 => connects port p1 on i1/i2/i3 to the net i1/i2/n1.

detachModulePort: Detaches the net connected to the specified port on the specified instance.
detachModulePort i1/i2/i3 p1 => detach port p1 from module i1/i2/i3

addNet: Adds a net to the design. The net can be logical or physical. Ex:
addNet i1/n1=> adds net i1/n1

addInst: Adds an instance and places it in the design. Ex:
addInst -cell BUF1 -inst i1/i2 -loc 100 200 => adds buffer instance i1/i2 at location 100, 200. (-cell specifies master of instance while -inst is the actual instance)

attachTerm: Attaches a terminal to a net. If the terminal already connects to a different net, the software first detaches the terminal from the current net, then attaches it to the new net. Previously we used to use: detachTerm to detach the existing net, but not needed any more. Ex:
attachTerm i1/i2/i3 in1 i1/i2/net26 => attaches terminal in1 of instance i1/i2/i3 (in1 is a port of i3) to net i1/i2/net26

NOTE: addModulePort and attachModulePort cmd can be avoided by using attachTerm which is more generic cmd.
#For ex, to connect internal gate o/p within one module to internal gate i/p within another module, use this:
attachTerm  spi_regs/eco2_inv A clock_reset_gen/n37 => attaches terminal A of inv in "spi_regs" module to net n37 in "clock_reset_gen" module. Note, this cmd first figures out the port thru which pin A of inv can be accessed, and then connects the net to that port. So, if there are multiple connections to that port, all of them will get conncted. In essence, this cmd connects a port to a net. If the port doesn't exist, it creates a port in the module (spi_regs) with that netname (n37).

#alternative way would be to have ports and then connect them
addModulePort     spi_regs  eco2_spi_inp input
addNet     -moduleBased digtop eco2_rst_connect
attachModulePort  spi_regs  eco2_spi_inp eco2_rst_connect => created port for "spi_regs" module and connected net to it.
addModulePort     clock_reset_gen eco2_clk_reset_out output
attachModulePort  clock_reset_gen eco2_clk_reset_out eco2_rst_connect => created port for "clock_reset_gen" and connected same net to it.
attachModulePort  clock_reset_gen eco2_clk_reset_out clock_reset_gen/n37 => connects the other end of port to net n37

Steps: to do ECO fix, first find the net name to where u want to insert ur logic. Use debussy on gate verilog and find the net on schematic of that module. If o/p net of new logic goes to fewer instances compared to i/p net of new logic, leave i/p net as existing net name and make o/p net as new net. Do vice versa (i.e if o/p goes to more instances, leave o/p net as existing net and make i/p net as new net)
Ex: attach an inverter to input of flop
#-moduleBased <module_defn_name> sets the module defn_name so that hier is not required. Note we specify module_defn_name and NOT module_instance_name. As module_name is unique for each instance in the synthesized netlist (see final synthesized netlist format desc above), it works OK.
addInst    -moduleBased spi_regs_test_1 -cell IV140 -inst eco1_inv_before => add inx instance
addNet     -moduleBased spi_regs_test_1 eco1_reg_input => add new net to connect to o/p of inx
attachTerm -moduleBased spi_regs_test_1 eco1_inv_before A n610 => attach inx i/p to existing net n610 (we chose i/p since n610 might be driving multiple loads)
attachTerm -moduleBased spi_regs_test_1 eco1_inv_before Y eco1_reg_input => attach inx o/p to new net.
attachTerm -moduleBased spi_regs_test_1 vbg2_op_out_reg_2 D eco1_reg_input => attach FF D i/p to new net. This causes exising net n610 to disconnect from D i/p of FF. If n610 wasn't connected to any other o/p, it would become floating.

---------------------
conformal ECO:
-------------------
Here, conformal generates the patch that can be used in VDIO. Run LEC:
lec -12.10-s400 -gui -xl -ecogxl -log ./logs/eco.log -dofile scripts/eco.do => -ecogxl enables post mask eco.

eco.do file:
-----
1. set common settings:
set log file eco.log -replace
usage -auto

2. Read library: (note that .liberty files are read, and NOT verilog models)
read library -both -liberty /db/pdkoa/.../MSL270_N_27_3_CORE.lib
read library -both -liberty /db/pdkoa/.../MSL270_N_27_3_CTS.lib  -append

3. Read design. Read 1p0 final PnR netlist as golden and new synthesized netlist as revised.
#NOTE: spare cell modules are missing from revised netlist since they are added in PnR flow
read design -verilog -golden  -sensitive -root digtop /db/HAMMER_OA/design1p0/HDL/FinalFiles/digtop/digtop_final_route.v
read design -verilog -revised -sensitive -root digtop /db/HAMMER_OA/design1p1/HDL/Syhnthesis/digtop/netlist/digtop_scan.v

4. set eco directives:
A. enable ECO meodeling directive. It's necessary to have this for eco. Other directives are optional.
 1. set flatten model -eco => prevent default flatten modeling from removing important info that is vital to correlate the ECO change back to original netlist. It's a macro that enables a number of related modeling options such as "set flatten model -keep_ignored_po -noremove_real_buffer" etc.
 2. set flatten model -enable_analyze_hier_compare => analyze hier bdry (module bdry) comp of flattened design. Needed to do hier comparison later.

B. other directives:
set flatten model -Latch_fold => if needed
set flatten model -seq_constant => if needed
set flatten model -gated_clock => if needed Gated clock control

#scan shift en turned off since scan chain may be different in new synthesized netlist. scan_mode still allowed both 0/1 values as scan_mode signal is just like any other logic signal.
add pin constraints 0 scan_en_in     -golden =>
add pin constraints 0 scan_en_in     -revised =>

#specify any new pins added/deleted at top level
#ex: new pin new_in2 added at top level digtop which also goes into submodule ctrl. so, we add this eco pin in both modules for golden (since they don't exist in golden. If we don't specify ports for modules/sub-modules, then tool is not able to add it for golden, and so reports them as unmapped points.). -input specifies i/p pin, while -output specifies output pin (default is input).
NOTE: We specify module definition name and NOT instance of module, as pin needs to be added on module defn.
add eco pin scan_out_iso  new_in2 -input -golden => use delete eco pin for deleting pins.
add eco pin scan_out_iso  new_out2 -output -golden => scan_out_iso is a submodule but still referenced as module since conformal flattens the design.
add eco pin clock_reset_gen_test_1 new_out2 -output -revised => clock_reset_gen_test_1 is the module defn name in revised netlist.

set mapping method -unreach

5. analyze hier bdry and do hier comaparison (lec mode). It should show non-eq points. Then create patch based on that.
set system mode lec
#while doing hier comp, hier_analyze.do file is generated which has all cmds for hier comparison. ecopins.do file is also generated which has eco cmd for adding pins to modules which need it in new netlist
analyze hier_compare -dofile hier_analyze.do -replace -constraints -verbose \
                     -threshold 0 -noexact_pin_match -noexact_module_match \
                     -eco_aware -input_output_pin_equivalence -function_pin_mapping -ecopin ecopins.do

add compared points -all
compare => should show non eq points
report statistics => reports

compare eco hierarchy => we break down comparison to sub-module level
report eco hierarchy -noneq -verbose
analyze eco -hierarchical patch.v -replace -ecopin_dofile ecopins.do -preserve_clock => creates a patch file which has only the gate changes needed for 1p1

6. apply patch, then optimize patch based on spare/GA cells, then write final netlist.
set system mode setup
dofile ecopins.do => add pins needed to modules.
#apply patch: -auto Automatically reads in and applies all patches that were created with the ANALYZE ECO cmd in the current session. (-keephierarchy specifies that the ECO changes will be put in a sub-module. Do not use this option, as that will cause problems in VDIO)
apply patch -auto => shows patch file being read and applied.

#### this section optional = to check if patch is good
set system mode lec
add compare point -all
#delete compare point A_REG[0] => To omit certain non-equiv points from eco analysis
compare  // this checks if patch if good before optimization, design should be equiv (1p0 vs patch)
write eco design -replace -newfile digtop_tmp_eco.v => IF we write out netlist, it will have separate eco modules which will have the new instances/connections. Later, after doing optimize patch, we get netlist which has no separate eco mdules, but the changes are within the existing modules. The netlist with no separate eco modules is the one that can be used in VDIO, else it will give an error for having extra modules.

NOTE: we could stop here and use the netlist generated above in VDIO. However, there are 2 problems. First, the netlist has extra modules, and secondly it has cells in eco_modules which may not be present in spare_cell module, so these will need to be substituted by cells which are actually present in spare_cell module. So, optimize patch step needed.

####
set system mode setup
###spare/GA cells added for Post-mask eco only. For pre-mask, ignore this section
add spare cell -freedcell => add any freed up cells to be used as spare cells
add spare cell -deffile  /db/HAMMER_OA/design1p0/HDL/FinalFiles/digtop/digtop_final_route.def -sparecell spr_*/spr* => this adds all spare cells to be used for eco
#add spare cell -deffile  /db/HAMMER_OA/design1p0/HDL/FinalFiles/digtop/digtop_final_route.def -sparecell GAFILL* => this adds all GA cells for eco
#delete spare cell -sparecell spr_*/spr_AN2* => This disables any specific spare cell that we don't want to use
report spare cell => This shows all avilable freed cells as well as spare cells
###

#optimize patch does the actual mapping to get new netlist generated
optimize patch -verbose  -usespare -workdir WORK \ => for postmask using spare/GA cells, use -usespare
-library "/db/pdkoa/.../MSL270_N_27_3_CORE.lib \
          /db/pdkoa/.../MSL270_N_27_3_CTS.lib" \
-netnaming eco_net_%d \ => within each module, new eco nets named as eco_net_1, eco_net_2, etc
-instancenaming eco_instance_%d \ => within each module, new eco instances named as eco_instance_1, eco_instance_2, etc
-rcexec "rc -12.20-s014" \ => version of rc to use
-sdc /db/HAMMER_OA/design1p1/HDL/DesignCompiler/digtop/sdc/hmr_constraints.sdc \
-def /db/HAMMER_OA/design1p0/HDL/FinalFiles/digtop/digtop_final_route.def \
-lef /db/pdkoa/.../vdio/lef/msl270_lbc7_tech_3layer.lef \
     /db/pdkoa/.../vdio/lef/msl270_lbc7_core_iso_2pin.lef  \
-mapscript mapping.tcl => This is optional and creates a mapping file which maps new eco cells with location aware spare/GA cells. this can be used in PnR, so that PnR doesn't have to do tedius process of mapping

#report eco changes -script -file test.script -replace => generates ECO inst set file that can be used directly in verplex(by using -script option) or EDI (by using -encounter option. it generates eco directive file)
report eco changes > eco_changes.rpt => reports eco chamges for each module (as new nets,instances,pins,etc)
write eco design -replace -newfile digtop_final_route_eco.v => new netlist can be tkdiff with old netlist to see differences.

exit

7. Now use the netlist generated above in Encounter to do place and route as in any eco flow. This new netlist above just has new instances added/deleted, but doesn't have the mapped spare cell isntance connection. This will be done in EDI. However, we will still need to modify the above netlist to add scan chain connection for any newly added flops.

---------------
DIFF the layout: After the ECO change, verify the differences to make sure, only desired metal layers got changed.
--------------
A. generate laff files for digital design for both 1p0 db and 1p1 db.

1. On cadence CIW (cmd/log window, NOT the lib mgr), goto TI_Utils->Translators->Physical->LAFF_out@TI. We get new pop up box (CDS2Laff).
2. Provide Form Template File name if you have any(.cds2laff.ctrl). This file has all the info in it, so that we don't need to type anything in boxes below. you need to load this file to save typing. .cds2laff.ctrl lools like this (resides in /proj/DRV9401/users/kagrawal/DRV9401 or anywhere else):
topcells
Hawkeye_digtop_1p1  digtop  layout
end
laffname   /sim/BENDER/pindi/assura/digtop_1p1.laff
nosystemlayers
layermap /data/pdk/lbc8/rev1/cdk446/4.6.a.19/doc/cds2laff.map
logfile /data/BENDER/users/pindi/BENDER/CDS2LAFF.LOG
signal label

If .cds2laff_1p1.ctrl file is not loaded, then run steps 3 to 7, and then save this template as .cds2laff.ctrl.
3. leave run dir as current (.).
4. choose cell type as "cellname" and Provide Library name (Hawkeye_digtop_1p1), cell name(digtop), view name (layout)
5. Provide Laff file name to write to (ex: digtop_1p1.laff).
6. choose layer map table => this comes from the pdk doc dir. Without this layer mappings will not be correct and we may get a "syntax error" on running difflay (for HPA07, it's /data/pdkoa/50hpa07/2012.11.13/cdk/itdb/doc/cds2laff.map)
6. choose signal type as "label". (leave "exclusive layer mapping" as ticked)
7. click "OK", and the digtop_1p1.laff file is generated in the dir mentioned. (choose yes and no for the first 2 options that pop up)

Repeat steps 1 thru 7 for digtop_1p0 design to generate digtop_1p0.laff (by changing .cds2laff_1p0.ctrl file appr)

NOTE: Look at CDS2LAFF.LOG file. At the very bottom, we should see 0 errors and 0 warnings.

B. Diff b/w digtop_1p0.laff and digtop_1p1.laff
Open difflayman tool (by typing difflayman on the unix command window). On the gui, provide the laff path for the 1p0 and 1p1 laff files, cell names as digtop, path to summary out file and log path file, and then click submit. A new window pops up. when it's done, then you can click on "view summary" to see what layers changed.

-------------------------------------

SPEF: this file has Res,Cap and other parasitic info for all nets in design
----

Standard Parasitic Exchange Format (SPEF) is an IEEE standard for representing parasitic data of wires in a chip in ASCII format. Resistance, capacitance and inductance of wires in a chip are known as parasitic data. SPEF is most popular specification for parasitic exchange between different tools of EDA domain during any phase of design.
The specification for SPEF is a part of standard 1481-1999 IEEE Standard for Integrated Circuit (IC) Delay and Power Calculation System.

General Syntax
--------------
A typical SPEF file will have 4 main sections:
¡V a header section,
¡V a name map section,
¡V a top level port section and
¡V the main parasitic description section.

Generally, SPEF keywords are preceded with a *. For example, *R_UNIT, *NAME_MAP and *D_NET.
Comments start anywhere on a line with // and run to the end of the line. Each line in a block of comments must start with //.


A. Header Information
---
The header section is 14 lines containing information about
¡V the design name,
¡V the parasitic extraction tool,
¡V naming styles
¡V and units.

When reading SPEF, it is important to check the header for units as they vary across tools.

ex: digtop.spef file:
--
*SPEF "IEEE 1481-1998"
*DESIGN "digtop" <design name
*DATE "Mon Mar 12 12:11:11 2012"
*VENDOR "Silicon Perspective, A Cadence Company"
*PROGRAM "Encounter"
*VERSION "09.12-s159_1"
*DESIGN_FLOW "COUPLING C" "PIN_CAP NONE" "NAME_SCOPE LOCAL"
*DIVIDER /
*DELIMITER :
*BUS_DELIMITER []
*T_UNIT 1 NS <= time units is ns
*C_UNIT 1 PF <= cap unit is pf
*R_UNIT 1 OHM <= res unit is ohm
*L_UNIT 1 HENRY <= ind unit is H

B. Name Map Section: optional
----
To reduce file size, SPEF allows long names to be mapped to shorter numbers preceded by a *. This mapping is defined in the name map section.
ex:
--
*NAME_MAP

*1 spr_6/FE_OFCN92_tie_hi_net0 <= net name mapped. Later in the file, it can be refrred by *1
*2 spr_6/FE_OFCN93_tie_hi_net0
*2246 trim_dout[52]
*3062 U57 <= instance name mapped

C. Port Section
-----
The port section is simply a list of the top level ports in a design. They are also annotated as input, output or bidirect with an I, O or B.
ex:
--
*PORTS

*1973 I *C 0 2069 => *1973 is n_puc i/p pin.
*1975 I *C 185.9 2073.6

D. Parasitics: this is the main section. It has info about each and every net is design.
----
Each extracted net will have a *D_NET section. This will usually consist of a *D_NET line, a *CONN section, a *CAP section, *RES section and a *END line. Single pin nets will not have a *RES section. Nets connected by abutting pins will not have a *CAP section.
The *D_NET line tells the net name and the net's total capacitance. This capacitance will be the sum of all the capacitances in the *CAP section.

ex:
--
*D_NET *1 0.163578 => net name is *1 and total cap on this net is 0.16pf = 160ff. Each net is subdivided into nodes with node id, which are used in cap and res fields below.

*CONN => lists all the pins connected to the net. A connection to a cell instance starts with a *I. A connection to a top level port starts with a *P. The syntax of the *CONN entries is:
*I <pin name> <direction> *C <xy coordinate> *L <load_pin_cap> *D <load_cell_type>

*I *2453:CLK I *C 493 1783 *L 0.00608 *D DTB20 => net *1 is connected to pin "clk" of instance *2453. clk is located at coord <493,1783>.  clk pin cap is 6ff and it's on cell DTB20.
...
*I *2323:A I *C 368 1943 *L 0.0129 *D BU140

*CAP => provides detailed capacitance information for the net. Entries in the *CAP section come in two forms, one for a capacitor lumped to ground and one for a coupled capacitor.
A capacitor lumped to ground has three fields: an identifying integer, a node name and the capacitance value of this node
A coupling capacitor has four fields: an identifying integer, two node names and the coupling capacitor values between these two nodes

1 *2323:A 9.27572e-05 => This lumped cap is given id 1, it's cap of 0.09ff to pin A of inst *2323
...
115 *2447:A *1707:33 0.000475296 => this coup cap is b/w pin A of *2447 and node id 33 of *1707(o/p buf x4).

*RES => provides the resistance network for the net. The resistance network for a net can be very complex.
Entries in *RES section contain 4 fields: an identifying integer, two node names and the resistance between these two nodes. Out of the 2 nodes, one of the node has to be the node for this net itself. The other node can be end point connection (any load in *conn) or other node on this net itself.

1 *1:19 *1:27 38.7104 => res of 38.7ohm b/w node id 19 and 27 of *1(this net).
...
46 *3565:Y *1:46 8 => res of 8ohm b/w node id 46 of this net and pin Y of *3565

*END => end of NET *1

*D_NET *2 0.0863349 <= repeat same as above for net *2
...
*END

*D_NET *2311 0.0154887 <= repeat same as above for all nets
...
*END

----------------------------------------------------
sample ex of a net:

*D_NET *2311 0.0154887 => net name *2311(wr_strobe_spi_sync). total cap=15.5ff which is the sum of all cap in *CAP. From digtop_route.v, we see that wr_strobe_spi_sync has following connections:
load1: NO210 U4 (.Y(n1), .B(n20), .A(wr_strobe_spi_sync));
driver: DTCD2 wr_strobe_spi_sync_reg (.Q(wr_strobe_spi_sync), .D(N28), .CLRZ(nreset), .CLK(clkosc__L2_N4));
load2: NO211 U161 (.Y(N28), .B(wr_strobe_spi_sync), .A(wr_strobe_spi_meta));
load3: IV110 U162 (.Y(n18), .A(wr_strobe_spi_sync));
load4: AN2D0 U176 (.Y(N145), .B(wr_strobe_spi_sync), .A(n193));

*CONN => there are 4 loads and 1 driver connected to this net.
*I *3556:B I *C 610 1416 *L 0.00265 *D AN2D0 => load4 is pin B of *3556(AN2D0). pin cap=2.6ff.
*I *3549:A I *C 540 1443 *L 0.00638 *D IV110 => load3 is pin A of *3549(IV110). pin cap=6.4ff.
*I *3548:B I *C 591 1416 *L 0.00561 *D NO211 => load2 is pin B of *3548(NO211). pin cap=5.6ff.
*I *3195:A I *C 588 1404 *L 0.00568 *D NO210 => load1 is pin B of *3195(NO210). pin cap=5.7ff.
*I *3220:Q O *C 622 1400 *L 0 *D DTCD2 => driver is pin Q of *3220(DTCD2). pin cap=0ff as it's o/p pin, even though in cap section, cap of 0.8+1.1+1=2.9ff is assigned to this o/p pin Q.

Now, this net (wr_strobe_spi_sync) is subdivided into subnets and node numbers are assigned to end points of these subnets. That forms a tree. Node ID appear in the connection part of cap and res section. If you look in connections of *RES section, you'll see that Node numbers are 5,6,7,8,9,10,11 (7 nodes) while the 11 res connecting these nodes and load/driver have been assigned 11 ids. Similarly in *CAP section, you will see that these 7 nodes are being used to connect coup/lump caps to, along with coup/lump caps attached to load, while 17 caps have been assigned 17 ids.
NOTE: Id of Res or cap have nothing to do with node numbers. Node numbers are the ones that form tree. Id for res and cap just indicate that particular cap/res were assigned some id. Same node numbers will appear in both res and cap section.
Below, a tree is formed as shown starting from driver o/p Q, and ending at 4 loads, ld1 to ld4. Node Ids 5 to 11 are shown.
Q->5->6->     10->     8->9->7->ld1
      6->ld4  10->ld2     9->11->ld3

*CAP => this section contains cap at each node and at some loads. Some loads are missing, not sure why?
top 10 are lumped cap, while 11 to 17 are coupling cap. These coupling cap are treated appr when running timing for setup/hold. In PT/ETS, all coupling cap are grounded to 0. So, it may be optimistic or pessimistic for setup/hold depending on which data or clock path it appears on. Ideally for setup, data path coup cap should be mult by 2, and clk path coup cap mult by 0, while for hold data path coup cap mult by 0, and clk path coup cap mult by 2. In PTSI, it does it that way.

1 *3220:Q 0.000805389 => driver o/p lump cap is 0.8ff. cap placed at o/p Q.
2 *3548:B 0.000700078 => lump cap placed at ld2
3 *3556:B 0.000671207 => lump cap placed at ld4
4 *2311:11 0.00116516 => lump cap placed at nodes 5 to 11
5 *2311:10 0.00159617
6 *2311:9 0.000124281
7 *2311:8 0.000102239
8 *2311:7 0.00367203
9 *2311:6 0.000747997
10 *2311:5 0.000117708
11 *2311:7 *3219:CLK 0.000676197 => coupling cap to clk of *3219 at node 7 of wr_strobe_spi_sync
12 *3220:Q *1929:10 0.00112872 => coupling cap to node 10 of *1929 placed at o/p Q.
13 *2311:7 *1762:7 0.000385412
14 *3556:B *170:8 0.000849882 => coupling cap to node 8 of *170 placed at ld4
15 *3556:B *3246:D 0.00124761 => coupling cap to pin D of *3246 placed at ld4
16 *2311:7 *1964:18 0.000400017
17 *3220:Q *1972:163 0.00109857 => coupling cap to node 163 of *1972 placed at o/p Q.

*RES => this section will contain all load and driver, as there has to be a res to the final end point. So, starting from driver, we can form a tree to all the loads:

1 *2311:8 *2311:10 8.80112
2 *3549:A *2311:7 17.2017
3 *2311:8 *2311:9 0.839437 => res b/w node id 8 and 9
4 *3556:B *2311:6 17.2017
5 *2311:9 *2311:11 11.3902
6 *3195:A *2311:11 8.16737 => res b/w node id 11 and ld3
7 *3548:B *2311:10 9.20169
8 *2311:5 *2311:6 2.04502
9 *2311:7 *2311:9 8.07294
10 *2311:6 *2311:10 11.2614
11 *3220:Q *2311:5 19.0042
*END

-----------------------------

UPF Syntax:

UPF is based on tcl scripting language. UPF commands are defined using syntax that is consistent with Tcl, such that a standard Tcl interpreter can be used to read and process UPF commands. Compliant UPF processors shall use Tcl version 8.4 or above.

UPF supports the specification of attributes, or properties, of objects in a design (eg UPF_clamp_value, etc). Many Liberty attributes are mapped to UPF attributes (i.e lib attr "pg_type" is mapped to UPF attr "UPF_pg_type).

There are 50 or so UPF cmds to specify power intent of design. We will deal with few imp ones. There are many legay or deprecated cmds which should not be used any more.

Power Domains, states and scope:

0. set_scope <inst_name> => This sets current scope of design. Inst name can be scope/design relative hier name.

  • slash = / => changes scope to top inst of design
  • dot = . => sets scope to current scope
  • double dots = ..  => sets scope 1 hier level higher, or parent of current scope.

If design has "top" as top inst, then "mid" and then "bot", then :

  • ex: set_scope top => This sets current scope to instance top.
  • ex: set_scope top/mid => sets current scope to mid

1. create power domains, port, nets and connect them: These cmds add the pwr signals to design.

A. create_power_domain <PD_NAME> => defines a power domain and the set of instances that are in the extent of the power domain. The scope of this PD is the current scope of design, so scope has to be set beforehand using "set_scope" cmd. options:

  • -supply: defines the supply sets (SS) that are used to provide power to instances within the extent of the power domain. The supported supply set handles are primary, default_retention, default_isolation, and extra_supplies_#. Primary is the primary SS for this PD, while extra_supplies_0, extra_supplies_1, etc are the power supplies that are needed for other smaller portions of the PD.
  • -elements: If -elements <list> option is used, then only those  inst (and all their descendants) specified in the list are in extent of that PD. option -exclude_elements excludes specified inst from the extent of this PD. By default, everything in scope is in extent of that PD. PD may contain elements from different hier of chip, as A/inst1, B/inst2, C/D/inst3 => so these 3 instances will be in one PD, even though logically they are in 3 different modules.

ex: create_power_domain PD_CHIP -include_scope -supply {primary {SS_VDD1p2}} -supply {extra_supplies {SS_VDDIO}}=> here supply set for primary is set to SS_VDD1p2, and extra_supplies to SS_VDDIO, for this PD. Here, scope is whatever scope was set to.

ex: create_power_domain PD_CHIP -elements {i1 i2 i3} -supply {primary {SS_VDD1p2}} => This includes instances i1, i2 , i3 and all their descendants in the PD_CHIP domain. No other instances from current scope are in this PD.

B. create_supply_port <PORT_NAME> => creates supply port at current scope. If option -domain <PD_NAME> is used, then PORT is created in the scope of speciifed PD.

C. create_supply_net <NET_NAME> => same as above, except that it creates supply net

D. create_supply_set <SET_NAME> => creates supply set. option -function {func_name net_name} defines function (func_name) that a supply net (net_name) provides for this set. Thus it maps each supply net in the set with 1 of 6 functions, These nets comprise the set.

ex: create_supply_set SS_VDD1p2 -function { power VDD1p2 } -function { ground VSS } -function { nwell VDD1p2 } -function { pwell VSS }

We can grab the supply net name from the supply set by using hier with dot as separator, i.e To get net name VDD1p2, we use "SS_VDD1p2.power". Similarly to get net name VSS, we use "SS_VDD1p2.ground".

E. connect_supply_net <NET_NAME> -ports <PORT_NAME> => connects supply net to supply ports

2. power/supply states: specifies various power states possible with various supplies. For ex, VDD supply might have 3 possible voltage values, so we may define 3 power states for it.

- add_power_state <OBJ_NAME> -state <STATE_NAME>  (add_port_state and add_pst_state, along with create_pst are legacy and shouldn't be used) => defines power state of an object which can be a PD, inst, supply net/port/set, etc. Usually power state is defined on supply set. If -supply is specified, the object_name shall be the name of a supply set or a supply set handle. option "-supply_expr {boolean_expr} " specifies a Boolean expression defined in terms of supply ports/nets/set, that evaluates to True when the object is in the state being defined. We can add as many power states as we want to <STATE_NAME> by repeatedly using this cmd.

ex: add_power_state PD_CHIP.primary -state { GO_ST -supply_expr {(power == {FULL_ON 0.8}) && (ground == {FULL_ON 0})} } -state { OFF_ST -supply_expr {power == OFF} } => defines 2 power states = GO_ST and OFF_ST, for PD_CHIP primary supply set which as defined above is SS_VDD1p2

ex: add_power_state SS_VDD1p2 -state V_NOM {-supply_expr {power == `{FULL_ON,0.5,0.6,0.7}}} => creates power state V_NOM for supply set "SS_VDD1p2".

- add_supply-state <OBJ_NAME> -state {<NAME>  <nom_val|off>} => defines a named supply state for a supply object which can be supply port/net/set. If a voltage value is specified, the supply net state is FULL_ON and the voltage value is the specified value; otherwise, if off is specified, the supply net state is OFF.

ex: add_supply_state PD.primary.power -state {active_state 0.90 } -state {off_state off} => defines supply states for condition when active_state is at 0.9V, while off_state is set to "off"

Power management cells: We need Power mgmt cells as power switch, level shifter, isolation, repeater and retention cells. there are set_* cmds to instantiate these in design (except for power switch which has create_* cmd), map_* cmds to map specific lib cells to such cells, and define_* cmds which are basically same as map_* cmds. Not sure why are there map_* and define_* cmds. Power switch, repeater and retention cells have corresponding map_* cmds (there's no map cmd for level shifter and isolation cells), while power switch, level shifter, isolation and retention cells have corresponding define_* cmds (repeater cells don't have a define_* cmd).

3. power switch: create and map/define pwr switch. create cmd used for power switch only (NOT for other cells, as set* cmd is used for those. This is because these are power strategies). Power switch is created just like ports/nets are created.

- create_power_switch <SW_NAME> -domain <PD_NAME> => defines a power switch in the scope of domain specified. This is abstract model of a single switch, impl may use multiple distributed pwr switches. Connections to i/p, o/p, ctl and ack port of switch is done via additional args here or using the map cmd (explained in example below). An on/off state condition is also defined, so that flows can understand how to turn the switch on or off.

ex: create_power_switch sw1 -domain PD_CHIP -control_port {SLEEPN_IN psw1/buf_in/y} -ack_port {SLEEPN_OUT psw1/buf_out/a} -on_state {SW_ON SLEEPN_IN} -off_state {SW_OFF !SLEEPN_IN} -input_supply_port {TVDD VDD_12} -output_supply_port {VDD VVDD_12} => This defines a power switch and connects the ports of power switch to various nets. Here control/ack ports of power switch are named as SLEEPN_IN and SLEEPN_OUT and are connected to some internal net in power switch wrapper. When SLEEPN=1 (i.e SLEEP=0), power switch is on. Vice-versa for when SLEEPN=0. The supply ports are also mapped to i/p supply and switched o/p supply.

- map_power_switch <SW_NAME> -lib_cells {LIB_CELL_LIST} -port_map {port_mappings} => This specifies list of lib cells to which impl of this switch can be mapped to. Lib cells should appear in liberty file with required "power switch" attr or such attr should be added via "define_power_switch_cell" upf cmd.

ex: map_power_switch switch_sw1 -domain test_suite -lib_cells {sw1} -port_map {{inp1 vin1} {inp2 vin2} {outp vout} {c1 ctrl_small} {c2 ctrl_large}}

- define_power_switch_cell -cells {cell_list}  => this cmd identifies the library cells that can be used as power switch in a design. Lots of options available with this cmd. This cmd not needed if "power switch" attr is specified on such cells in liberty files.

4. retention cell: set and map/define retention cells. These reg/flops already exist in RTL design, just that we identify which of these are going to be retention, and what strategy is going to be used.

- set_retention <RET_NAME> -domain <PD_NAME> => specifies retention strategy, and the domain to which it's applied. Other options provide save/restore ports and their active level/edge for save/restore to happen, save/restore condition (logic needed to be connected to generate save/restore condition). option -elements or -exclude_elements can apply this to specific elements instead of applying it to whole PD. -retention_supply provides supply set used to power the retention cells.

- map_retention_cell <RET_NAME> -lib_Cells {LIB_CELL_LIST} -port_map {port_mappings} => This speciifes list of lib cells to which these retention cells can be mapped to. Lib cells should appear in liberty file with required "retention" attr or such attr should be added via "define_retention_cell"

ex: map_retention_cell {my_PDA_ret_strat_1 my_PDA_ret_strat_2} -domain PD_A -elements {foo/U1 foo/U2} -lib_cells {RETFFIMP1 RETFFIMP2}  -port_map { {CP UPF_GENERIC_CLOCK} {D UPF_GENERIC_DATA} {SET UPF_GENERIC_ASYNC_LOAD} {SAVE save_signal} {RESTORE restore_signal} {VDDC primary_supply.power} {VDDRET retention_supply.power} {VSS primary_supply.ground} }

- define_retention_cell -cells {cell_list}  => this cmd identifies the library cells that can be used for retention in a design. Lots of options available with this cmd. This cmd not needed if "retention" attr is specified on such cells in liberty files.

5. repeater cell: set and map repeater cells. map cmd for repeater cells doesn't have an equivalent define_* cmd.

- set_repeater <REP_NAME> -domain <PD_NAME> -applies_to <Inputs|outputs|both> -repeater_supply <SUPPLY_SET> => specifies repeater strategy, and the domain to which it's applied. supply_set that powers this buffer is specified.

- map_repeater_cell <REP_NAME>  -domain <PD_NAME> -lib_cells {LIB_CELL_LIST} => This speciifes list of lib cells to which these repeater cells can be mapped to. NOTE: there is no "repeater" attr needed on lib cells or "define repeater cells" upf cmd, as these are regular buffer cells.

ex: map_repeater_cell my_rep1_pd1 -domain PD1 -elements { clk1 rst1 clkout1 rstout1 } -lib_cells { aon_clk_bufx2 }

6. level shifter cell: set and define level shifter strategy. It's applied at the domain boundary, as required to correct for voltage differences between driving and receiving supplies of a port. There's no map cmd for level shifter in UPF IEEE doc, though it's used as valid UPF cmd everywhere.

- set_level_shifter <LVL_NAME> -domain <PD_NAME> -applies_to <Inputs|outputs|both> -rule <low_to_high|high_to_low|both> -input_supply <IN_SUPPLY_SET> -output_supply <OUT_SUPPLY_SET> => defines level shifting strategy for ports on i/f of specified PD. port_dirn to which this applies can be specified as i/p ports, o/p ports or both (by default, strategy applied to all ports). Rule specifies if level shifter strategy should be applied only to ports needing level shifting in one dirn (default is to apply it to ports needing level shifting in any dir, i.e high to low or low to high). option -no_shift can be used to not apply level shifter on specified ports.

ex: set_level_shifter DFT_input_ls -domain PD_DFT -applies_to inputs -rule both -location parent => defines Level shifter on all i/p ports of PD_DFT power domain.

ex: set_level_shifter DFT_no_ls -domain PD_DFT -no_shift -elements {mux0/out1 {mux2/In[*]} mux3/in3} => This cmd specifies no level shifter on these ports. So, all i/p ports in above cmd will have level shifter, except for ports specified here.

- define_level_shifter_cell -cells {cell_list} -enable <EN_PIN> -direction <low_to_high|high_to_low|both> => this cmd identifies the library cells to use as level-shifter cells. This cmd replaces "map level shifter cells", as all lib cels defined here can be mapped to level shifter strategy defined. 

7. Isolation cell: set and define isolation strategy.Just like level shifters, it's applied to ports at the domain boundary, so that correct electrical and logical functionality is maintained when domains are in different power states. Again, there's no map cmd for isolation cells in UPF IEEE doc, 

- set_isolation <ISO_NAME> -domain <PD_NAME> -applies_to <Inputs|outputs|both> -clamp_value <0|1|Z||latch|value> -isolation_signal {iso_ctl_signal} -isolation_sense <high|low> -isolation_supply <supply_set_list>=> specifies isolation strategy, and the domain to which it's applied. Isolation ctl signal of cell, as well as the active level of ctl signal is specified. We can also specify the Supply set for isolation cell.

ex: set_isolation PD_efuse_in_iso -domain PD_EFUSE -applies_to inputs -clamp_value 0 -isolation_signal pgctl_clamp -isolation_sense high -isolation_supply SS_VDD_1 

- define_isolation_cell -cells {cell_list} -enable <EN_PIN> -clamp_cell <high|low> => this cmd identifies the library cells that can be used for isolation in a design. This cmd replaces "map iso cells", as all lib cels defined here can be mapped to iso strategy defined. Iso cells can be of 2 types => clamp type (clamps o/p to high or low when EN_PIN is active) or non-clamp type (cell function determines cell o/p).

ex: define_isolation_cell -cells iso_cell1 -power VDD -ground GVSS -enable iso_en

8. Port attributes: specifies info associated with ports. These port attributes identifies port's related supplies, and aid in isolation and level shifting insertion.

- set_port_attributes -ports {port list} -clamp_value <0|1|Z|value> -driver_supply {driver supply set} -receiver supply {receiver supply set} => clamp_vale specifies the clamp value to be used if an isolation strategy is applied to the port. Driver_supply is used for an o/p port to specify the driver supply of logic driving the o/p port. Similarly for i/p port, we have receiver supply.

ex: set_port_attributes -ports {port1 port2[*]} -driver_supply SS_VDD12 -receiver supply SS_VDD12 => Here driver and receiver supply for all these ports are set to same supply voltage, implying all ports of current instance.


UPF cmd file Example: (ex in http://rd.springer.com/chapter/10.1007/978-1-4614-4271-4_8/fulltext.html)

In the example above, there are 3 power supplies and 4 power domains defined.

3 pwr supplies => (PVDDdsp=1.1V_0.9V, PVDD1p0=1.0V, PVDD0p9=0.9V), but 4 pwr domains, since PD_COP has a switch to connect to 1.0V spply
4 pwr domains =>

  • PD_MYCHIP=1V,
  • PD_CPU=0.9v (both always ON)
  • PD_COP=1V (from PD_MYCHIP, but can be shutdown using a switch)
  • PD_DSP=1.1V or 0.9V

We can define various power modes (basically 4 power modes, where PD_COP may be ON or OFF, and PD_DSP may be 1.1V or 0.9V), and other 2 PD are always ON.

There is a power ctl in PD_MYCHIP domain which controls switch, drives isolation and state retention signals to PD_COP.

mychip.upf:
------
# Set scope to top-level:
set scope

# Declare power domains: (create for all 4 PD)
create_power_domain PD_MYCHIP - include_scope => creates power domains
create_power_domain PD_CPU - elements {U_CPU} => and so on for other 2

# Create power nets at top : (similarly do for other 3 PD => PD_CPU and PD_DSP have only 2 supply nets, one of VDD* and GND, PD_COP has 3 supply nets: VDD1p0, VDD1p0_SW and GND. VDD1p0 is needed since it has retention flops which are always ON)
create_supply_net VDD1p0 - domain PD_MYCHIP - reuse => nets can be any name. supply_net is needed to connect supply ports
create_supply_net VDDdsp - domain PD_MYCHIP - reuse
create_supply_net VDD0p9 - domain PD_MYCHIP - reuse
create_supply_net GND    - domain PD_MYCHIP - reuse

# Create the power ports at top:(similarly do for other 3 PD, all of which have 2 ports). For each PD, supply and gnd port must be specified
create_supply_port PVDD1p0 - domain PD_MYCHIP => port names have to be top level port names
create_supply_port PVDD0p9 - domain PD_MYCHIP
create_supply_port PVDDdsp - domain PD_MYCHIP
create_supply_port PGND - domain PD_MYCHIP

#Connect top power ports and nets: => all ports connected to internal nets
connect_supply_net VDD1p0 - ports PVDD1p0 => this connects the supply ports created
connect_supply_net VDD0p9 - ports PVDD0p9
connect_supply_net VDDdsp - ports PVDDdsp
connect_supply_net GND - ports PGND

#connect top to PD_CPU (connects each lower level module port to these nets) => similarly for other 2 PD
connect_supply_net VDD0p9 - ports {U_CPU/PDCPU_VDD0p9}
connect_supply_net GND - ports {U_CPU/PDCPU_GND}

# Connect inside PD_CPU: => similarly for other 2 PD
connect_supply_net VDD0p9 - ports PDCPU_VDD0p9 - domain PD_CPU
connect_supply_net GND - ports PDCPU_GND - domain PD_CPU

# Specify primary power nets:  It specifies one primary power and ground connection for every power domain.
set_domain_supply_net PD_MYCHIP - primary_power_net VDD1p0 - primary_ground_net GND
set_domain_supply_net PD_CPU - primary_power_net VDD0p9 - primary_ground_net GND
set_domain_supply_net PD_DSP - primary_power_net VDDdsp - primary_ground_net GND
set_domain_supply_net PD_COP - primary_power_net VDD1p0_SW - primary_ground_net GND

#####
#Define isolation strategy and control for PD_COP:
set_isolation PD_COP_ISO \
- domain PD_COP \
- isolation_power_net VDD1p0 \ => supply nets for isolation logic
- isolation_ground_net GND \
- applies_to outputs \ => isolation only applies to output ports of this PD
- clamp_value 0 => specifies value to which iso i/p or o/p are clamped. Here it's iso low

set_isolation_control PD_COP_ISO \ => every set_isolation cmd should have corresponding set_isolation_control
- domain PD_COP \
- isolation_signal U_PC/ISE \ => This can only be a net (no port or pin)
- isolation_sense low \
- location self => self means iso cell is placed within current hier, while "parent" implies it's placed in parent module.

#####
# Define level shifter strategy and control for PD_CPU: (similarly for PD_DSP) => needed only when voltage levels differ
set_level_shifter FROM_PD_CPU_LST \
- domain PD_CPU \
- applies_to outputs \
- rule low_to_high \
- location parent

set_level_shifter TO_PD_CPU_LST \
- domain PD_CPU \
- applies_to inputs \
- rule high_to_low \
- location self

# Declare the switches for PD_COP:
create_power_switch PD_COP_SW \
- domain PD_COP \
- input_supply_port {VDDG VDD1p0} \ => 1st arg is port name, 2nd arg is net name connecting to it.
- output_supply_port {VDD VDD1p0_SW} \
- control_port {SLEEP U_PC/PSE} \
- ack_port {SLEEPOUT U_PC/PSE_ACK} \ => ACK o/p port if present
- ack_delay {SLEEPOUT 10} \ => delay from CTRL i/p port to ACK port
- on_state {SW_on VDDG !SLEEP} \ => SW_on state is defined as when SLEEP port is low
- off_state {SW_off SLEEP} => SW_off state is defined as when SLEEP port is high

# Specify the switch type: This command specifies which cell to use from a technology library for a power switch.
map_power_switch PF_COP_SW \
- domain PD_COP \
- lib_cells HEADBUF_T50

# Specify isolation cell type:
map_isolation_cell PD_COP_ISO \
- domain PD_COP \
- lib_cells {O2ISO_T50 A2ISO_T50}

##########
# Specify retention strategy: (i.e which reg in that PD are to be implemented as ret reg)
set_retention PD_COP_RET \
- domain PD_COP \
- retention_power_net VDD1p0 \ => ret pwr/gnd net are connected to save/restore logic and shadow regs.
- retention_ground_net GND \
- elements {U_COP/reg1 U_COP/pc U_COP/int_state} => All reg here are given ret capability. If no elements specified then element list used to define PD is used.

set_retention_control PD_COP_RET \ => each ret strategy has corresponding ret control.
- domain PD_COP \
- save_signal {U_PC/SRE high} \ => specifies net used to save data in shadow reg, and logic state of save signal that causes this to happen
- restore_signal {U_PC/SRE low} => same signal used for restore signal

map_retention cell MULT_RET \
-domain PD_COP \
-lib_cells RSDFF_X40

#########
# Add port state:
add_port_state PVDD1p0 -state {S1p0 1.0}
add_port_state PVDD0p9 -state {S0p9 0.9}
add_port_state PVDDdsp -state {SH1p1 1.1} -state {SL1p1 0.9}
add_port_state PGND -state {default 0}
add_port_state PD_COP_SW/VDD -state {SW_on 1.0} -state {SW_off off}

# Create power states and state table:
create_pst MYCHIP_pst - supplies \
{VDD1p0 VDD0p9 VDDdsp PD_COP_SW/VDD}

add_pst_state PM1 - pst MYCHIP_pst - state \
{S1p0 S0p9 SH1p1 SW_on}

add_pst_state PM2 - pst MYCHIP_pst - state \
{S1p0 S0p9 SH1p1 SW_off}

add_pst_state PM3 - pst MYCHIP_pst - state \
{S1p0 S0p9 SL1p1 SW_on}

add_pst_state PM4 - pst MYCHIP_pst - state \
{S1p0 S0p9 SL1p1 SW_off}

-------------------------------------

TCL = Tool cmd language

Tcl is similar to other shell lang. TCL can be scripting language, but it can also be used as shell as "tclsh".

TK can be used to develop gui i/f.
Tcl/Tk is open source, and can be included in C/C++ etc, or vice versa.

official tcl/tk documentation (very detailed): http://www.tcl.tk/doc/

As alwyas, tutorial point is very good source for learning anything: https://www.tutorialspoint.com/tcl-tk/tcl_tk_quick_guide.htm

TCL is a very weird language, interpretor written ver differently than how other high level lnguages are written in. Richard M Stallman has openly said that Tcl as a language should be completely avoided. There are weird cases, and language is more of learning by experience than learning by syntax. Same code written in 2 ways doing the same thing and syntactically correct, may be interpreted by the interpreter different and give different results. You should avoid using Tcl for any general purpose work, and use it only where it's absolutely needed for the job.

TCL is mostly used in CAD tools. If you are going to be using any hardware CAD programs from Synopsys, Cadence etc, they exclusively use tcl. So, if you have to learn it, whether you like it or hate it.

Tcl/Tk Installation:

tcl/tk may not be installed by default on most Linux OS. Here's the steps for downloading and nstalling tcl/tk:

On CentOS: With any software installation, there are 2 ways to install. Install it using package installer (which will automatically download and install all dependencies), or download it yourself and install it. The preferred way is using "yum" package installer on CentOS, but sometimes that package is not available via yum, so we may be forced to do manual process.

A. yum installation: sudo yum install tcl => This shows tcl.x86_64 as the pkg being installed. Click "Y" and it installs it in /usr/bin/tclsh (using cmd "which tclsh" shows it). For some reason, this wasn't working well with some other tool (missing files), so I installed it directly

B. direct installation: Download it from "https://www.tcl.tk/software/tcltk/download.html". I downloaded tcl8.7a1-src.tar.gz and tk8.7a1-src.tar.gz. After downloading it in some dir, goto that dir using gui file browser, right click and choose "extract". This will unzip and untar all files/dir and create new dir with same name "tcl8.7a1" and "tk8.7a1". Once this is done, goto unix terminal. 

1. cd tcl8.7a1/unix.

2. type "./configure"

3. type "make" => this will finish, with no "success" or "fail" info. Just some "gcc -O2 ..." cmds at the end.

4. type "make test" => runs all tests. Summary should show all passed (Total=31405, Passed=30101, skipped=1303, Failed=1= http.test). Many will be skipped. That's OK.

5. type "sudo make install" => installs it in "/usr/local/bin/tclsh8.7" All files are in /usr/local. Lib are in /usr/local/lib/tcl8.7/

6. If we run tclsh, it won't find anything. Even though "echo $PATH" shows /usr/local/bin in the path variable. Reason, tclsh is installed as tclsh8.7. We will fix it in step 8.

7. Now repeat same process for tk8.7a1. cd unix, ./configure, make, make test (Total=9544, Passed=8459, Skipped=996, Failed=89,Then again shows Total=465, Passed=448, skipped=17, failed=0), sudo make install. This gets installed as /usr/local/bin/wish8.7

 8. Now create soft links so that tclsh (which is actually tcl) and wish (which is actually tk) can run by typing tclsh or wish. cd /usr/local/bin. "ln -s tclsh8.7 tclsh" and "ln -s wish8.7 wish". Now all the tools which rely on tclsh or wish should be able to access it, as long as the path "/usr/local/bin" exists in PATH var. tclsh and wish are installed with their version numbers, so that different versions of the same lang can be installed at the same time without overwriting each other. That way if 2 different tools require 2 different versions of a pgm, they can run simultaneously by changing soft links or hardcoding the actual path of pgm with correct version number.

NOTE: when havving multiple versions of tcl/tk on your system, some files do not get installed correctly, resulting in "missing files" when calling an app that requires tcl/tk. Part of the confusion may be because some files are named same for for any version of tcl/tk installation resulting in 1 file getting used for mutiple versions, and depending on which is the last one that got over written, it may or may not work with other versions. Make sure you have these files:

> whereis tk
tk: /usr/lib64/tk8.5  /usr/include/tk.h /usr/share/tk8.5
> whereis tcl
tcl: /usr/lib64/tcl8.5 /usr/include/tcl.h /usr/share/tcl8.5

> whereis tclsh
tclsh: /usr/bin/tclsh /usr/bin/tclsh8.5 => tclsh is just a link to tclsh8.5

Depending on manual installation, these files might also be in /usr/local/bin/ and /usr/local/lib/ (since all software gets installed in /usr/local/ by default). Make sure, these files are there:

/usr/share/tcl8.5/ => has tclIndex, bunch of other tcl files and dir

/usr/lib64/  => has tclConfig.sh, libtcl8.5.so and tcl8.5 dir (this dir has tclConfig.sh which is just a soft link to tclConfig.sh in /usr/lib64/).

/usr/include/ => has tcl.h and bunch of tcl*.h files and dir


TCL syntax:

Any language has data and cmds (or operators) to act on data. Then there are special characters. All these are parsed by the interpreter to figure out what to do with that line to produce the result. We'll learn about data (var and their data types), cmd (reserved keywords and operators) and special characters (anything other than alphanumeric and operators).

tcl is case sensitive lang.

tcl script is a string consisting of 1 or more cmds. Tcl cmds are list of words, with first word being the cmd, and remaining words being args. This is totally different than how all other scripting languages or high level languages work. Whitespace separates multiple words of the cmd. To group multiple words into single arg, we use {} or "". To separate different cmds from each other, newline is used (so use separate line for each cmd). To put multiple cmds in same line, use semicolon ( ; ). So, newline and semicolon are very important in tcl (you need atleast 1 of them), else diff cmds may not be recognized.

NOTE: Since each cmd in tcl is on it's own line (or separated by ;), you can't put characters as {, }, ", etc anywhere. You have to follow strict syntax so that they get recognized as a group. So, tcl is very different than other languages where spaces, newline and positioning of characters matter. This is due to the fact that every string is treated as a cmd. So, if the string at the start of the line by itself doesn't seem to be a cmd, tcl will error out with "unkown cmd" (i.e having { by itself on a line is not a valid cmd). See more examples below with various cmds.

This link neatly explains how tcl interpreter interprets words: http://www.tcl.tk/man/tcl8.5/TclCmd/Tcl.htm

 

Tcl special characters: Just like all other languages, tcl has it's own set of special characters. They are mostly consistent with shell (bash, csh) special characters, but differ in some ways.

  • $ : dereferences a variable. $a refers to contents of var "a"
  • ( ) : Used for grouping expressions. This is used with arrays. It's not used in grouping cmds or words, as in "for" loop, etc. Curly braces { ... } explained below are used for all those cases. So, use of () is very limited in tcl.
  • [ ] : This does cmd substitution. Denotes a nested command. these evaluate cmds/scripts inside [ ]. The result is then put out and used by other cmds surrounding this. It means evaluate that string as a script immediately and use the result of the script as the value to substitute. It's used when calling user defined proc or other built in cmds.
  • \ : Used for escape quoting. Any char after \ is treated as ordinary char (loses it's special char meaning), and included in the word. Escape rule is similar to \ in bash, where it hides all special char from shell, but only 1 char at a time. The exception to this is the following special sequence => \a, \b, \t, \n, \\ ( this gets substituted as backslash \ ) , \xhh, \uhhhh,. So, \n is not escaped as tcl compiler sees the word "n" after "\" and omits escaping. However \\n will be seen as escape "\" followed by "\" and then a letter n, so final escaped value would be "\n". Escaping still happens within double quotes "..." but not in braces {...}. See next 2 bullets. 
    • If you put \ at end of line then since backslash escapes whatever comes after it, it will escape the newline. This looks similar to \\n, but here "\n" is one char (newline char), so that whole newline char is escaped. So, the shell doesn't see it as end of line. That means that next line will be treated as continuation of current line, and keeps on doing it for as many lines as you have \. All of these lines are then treated as one string.
    • VERY IMP: If you put space after a "\" at the end of current line, then \ escapes the space and not the newline. Escaping the space does nothing (since space is not a special char). Since newline is not escaped, the current line ends and new line starts from next line. This may break your code, as you may have intended for next line to be continuation of current line. Next line may not be valid cmd anymore and you may get "error". This is very common mistake in tcl, and since most editors don't show space explicitly, be extremely careful when using "\" at end of line. This is an issue in comment line starting with # too, see below for details.
  • " " : Denotes weak quoting; Command substitution, variable substitution, and backslash substitution are performed on the characters between the quotes. Groups multiple words into one. This is similar to  double quotes " " in bash, where only whitespace char are hidden from tclsh. It treats groups of words inside " " as 1 string.
    • ex: puts "..\n... \$a..." => Here newline char is not escaped (as it's excluded from escaping anyway(see escape quoting above). $ sign is escaped here even though it's inside quotes, as backslash escaping is still done inside quotes. So, we print $a instead of value of $a as $ got ecaped.
  • { } : Denotes rigid quoting; there are no substitutions. Groups multiple words into one. it means “don't do anything with this string inside { } at all; use it as-is. Sometimes you're feeding that string into a command that evaluates, so this helps do that, as nothing gets substituted. This is similar to single quotes ' ' in bash, where all special char are hidden from shell. The second most popular use of { } is to group words everywhere, i.e in loop body, conditional stmt, proc, etc. However, strictly speaking, they are not required at all except for places where we don't want substitution. We use them in proc args/body, but we can write proc entirely w/o using curly braces. curly braces within a comment are considered too when looking for start-end pair of {...}, so be very careful to not have single curly braces inside a comment (see below in comment section).
  • ; : Ends a command. needed when multiple cmds are used on same line
  • # :Begins a comment. Comments may be placed at start of line, or after end of cmd (which ends by having a ; at the end. Comments in Tcl are just not ignored, but are put in a "process later" bucket while the parser keeps moving forward. In tcl, you just can't take a piece of code, and comment it using "#" expecting it to run without errors. Syntax errors within "#" may still give errors. 
    • ex: set a 2; #my comment here => This works as a comment
    • if {0} { all comments } => #comment has many caveats as shown below, so one popular alternative to using # is to put comments in this "if 0 { ... }" construct (braces not necessary around 0). This always works without issues.
    • CAVEAT 1: braces {...} should always appear in pair in a comment, else it will give rise to mismatched { ... }. {...} in comments which are not in a pair will be errored out by the interpretor as not having matching braces. Also, the interpreter will point you to "error" in start of the program or in other weird places, when the error is due to mismatched { ... } in comments. It has no clue of where the mismatch happened, when it's nside a comment.
    • CAVEAT 2: comments in tcl are not just ignored, but are also looked for "\" at end of line. This is in contrast to bash/csh, where they are just ignored. So, if you comment out an existing piece of code which has \ at the end of line, your comment may not work. This is a pain, as comments can't be put blindly over a piece of code. One side benfit of this is that a comment can be extended to other line by using "\" at end of 1st comment line. This gives rise to some weird code as seen below:

ex: sometimes we see code where we execute a script2 within a shell script1, which sources the same original script1. This looks like a recursive infinite loop, but can work well using "\" at end of comment for tcl files. This script myscript.sh below is a bash shell script. Only 1st 3 lines are bash cmds, remaining are tcl cmds. On running, this script starts running as bash, based on 1st line, ignores 2nd line which is a comment (ignores \ at end of comment), and runs 3rd line "exec" cmd. That calls a pgm pt_shell which has a source $0 in its arg. pt_shell replaces the current shell and sources myscript.sh (which is the original file itself). However, pt_shell is a pgm that only sources tcl file, and starts parsing it per tcl syntax. It sees any line starting with # as comment . So, it ignores first line as comment (even though tclsh would have interpreted that, but looks pt_shell just ignores it). It looks at 2nd line, sees continuation of it on 3rd line (due to \ at end of 2nd line), and so considers 2nd line as comment (with 3rd line included in it). Then it moves to 4th line and start reading the file normally. So, here we are able to use the same file as both script and source file.

#!/bin/sh

# \

exec pt_shell "source $0"

other tcl cmds ....

variables:

Just like any other language, we define variables in tcl. Variables can be any data type. Tcl has it's own data type explained below. variable names need to conform to a syntax. In tcl, these var names or identifiers start with letter or _, followed by letters, digits, $,_. ex: a123_. However going thru tcl doc, it looks like a standard scalar variable can be comprised of any character. However the variable substitution operator "$" assumes you are using alphanumeric characters and underscores (plus name space separators "::"). So, we stick with _ and alphanumeric char for var names, so that they work w/o issues when used with $. To assign a value to a var, "set" cmd is used. To print value of any var, "puts" cmd is used. syntax of various cmds is explained in cmds section below.

ex: set a123! me => even though a123! has special char "!" in it's var name, tcl accepts it. It set var "a123!" to value "me"

puts $a123! => errors out, since $ assumes that var name is only _ and alphanumeric char. So, the first time it sees "!", it assumes that var name is complete and so var name is a123. ! is assumed to be start of next word. So, it tries to print value of var "a123", which obviously doesn't exist. Gives ERROR: can't read "a123": no such variable

If we put var inside curly braces, then no substitution is done, and $ works correctly.

puts ${a123!} => prints "me", since a123! is not substituted, but treated literally. So, $ sees a123! as var name. This technique also used when printing var concatenated to other char as "/a/b/${myvar}_c/d". Here, without curly braces, $ will look for var name "myvar_c" which doesn't exist.

unset a123! => This unsets the var so that it isn't defined anymore. However, if var a123! doesn't exist, then it will error out. So, the var needs to be set before it can be unset.

NOTE:  TCL is used almost exclusively with all CAD software to write scripts to be run on the CAD software. One thing to keep in mind is that when you run the same script multiple times or different scripts in the same session, var definitions from one iteration of the script or from multiple scripts are saved in memory, so the value of var from prior script or different script may still be used in current run. In such cases, the script may fail when run on a different session. Or you may get errors like "var myvar is not an array", when you want to set an array. This happens because var may be defined as normal var in previous iteration, and in latest run you are trying to make it an array, which conflicts with previous defn. In such cases you should always unset all var at beginnng of script, so that script starts clean. One issue is that var needs to be set before it can be unset. To get over that use catch.

ex: catch {unset a123!} => Now even if the var is not defined or set, it will not error out.

------------

In TCL there are many reserved words that can't be used as identifiers. ex: for, if, expr, etc. Also, tcl special var explained below can't be used as identifiers.

tcl special var: have predefined usage (i.e tcl_version, env, argc etc)

  • puts $tcl_version => gives 8.6
  • puts $env(PATH) => prints PATH = /home/cg/root:/..../:
  • argc = num of cmd line args, argv = list of cmd line args, argv0 = name of script being invoked

whitespace: whitespace (blanks, tab, etc) is needed in tcl to separate parts of stmt so that interpreter can identify them. Sometimes, it's not necessary, i.e [expr 3+2] will work correctly even though 3+2 has no whitespace. Recommended to use whitespace everywhere, so that there's no confusion.

Example script in tcl:

run standalone:

ex: hello.tcl
#!/usr/bin/tclsh => tells this script should be interpreted using tclsh.
# comment is here => comment. multiline comment would be using "if 0 { comments }". We do not have C style of multi line comment.
puts "Hello World!" => io cmd. puts is cmd, "Hello World" is arg. no semicolon at end. semicolon is needed if inline comment is used => puts "abc"; #comment.
puts stdout "Hello" => here 2 args provided. prints Hello on screen (std o/p)

run: ./hello.tcl or tclsh hello.tcl. If 1st line #! is not there in tcl script, then ./hello.tcl doesn't work (as current shell doesn't know what interpretor to use to run this file). Note: extension of file doesn't have any meaning in linux.

run interactively:

Instead of writing script, we can run tclsh in interactive mode by typing tclsh. Then shell changes from whatever shell you are currently in to tclsh. You can type tcl cmds in that shell, and it will show corresponding outputs. To exit tclsh, type exit and enter.

ex: [/home/ajay]$ tclsh => causes it to switch to tclsh
% expr 2 + 3 => n tclsh, we issue tcl cmds
5 => result of tcl cmd

% exit => cmd to exit tclsh

[/home/ajay]$ => after typing exit above, we are back to bash or csh


Data and Cmd:

Data may be integer, string,etc and have a identifier name (as my_name, etc). Cmds (or operators) are +,-,while, for etc. There are reserved words that are used for cmds which can't be used as identifier names. In many languages (eg C) data types need to be defined as to whether it's an integer, float, string, etc, while in most interpreting/shell languages (eg tcl) data types do not need to be defined, and are figured out automatically by the interpretor. This can be a source of confusion, as we do not know if a particular variable is treated as string or number in a particular operation. Many times, it's clear as some operators only operate on particular data tpes (i.e + only works on int and float), but sometimes it's more complex.

Data types: primitive data type is string (tcl is called string only lang). Everything is a string, and converted to other types as needed. No declaration needed. There are 2 composite (or derived) data types: list and array. So, this keeps the languauge very simple. Tcl can auto transform one data type to another when needed. However this makes the lnguage slow, as adding 1000's of numbers will require converting numbers from strings to number (as all numbers are by default treated as string, and then interpreted to be numbers which requires expensive conversion).

Following are the data types in tcl:


1. string: This is the only primitive data type. string can be put w/o quotes (for single words), with double quotes (for multiple words where we want weak quoting, i.e cmd, var and backslash substitutions allowed), or with curly braces (for multiple words where we want strong quoting, i.e no substitutions allowed).
ex: set myVariable 18 => stored as string, even though no quotes used, since it's single word.
puts [expr $myVariable + 6 + 9] => myVariable is converted to integer and 33 is returned, which is again stored as string and passed on to puts, for puts to print it.

ex: set me "John Hass" => double quotes used since it's more than 1 word assigned to var "me"

ex: set me {John Hass} => same as above.

string operations: manipulate strings. Lots of such cmds available. Listing a few below. they all need to have keyword "string" before using the operation cmd

compare: string compare string1 string1 => returns 0 if string1 equals string2.

ex: if {[string compare $s2 $s3] == 0} { puts "Strings same"; }

equal: string equal $s1 $s2 => returns 1 if both strings are same

map: string map {abc 1 ab 2 a 3 1 0} 1abcaababc => replaces substring in string "1abcaababc" with key value pair. So, abc is replaced by 1, ab is replaced by 2 and so on. Result is 01321.

index: various cmds to get index of string
ex: string first string_to_find string1 => finds first occurreence of string "first string_to_find" in string "string1". Similarly we have "last" cmd to find last occurrence of string.
ex: string index $s1 4 => finds character at index 4 of string $s1

ex: string length $string1 => finds length of $string1

subst: This command performs variable substitutions, command substitutions, and backslash substitutions on its string argument and returns the fully-substituted result. If any of the -nobackslashes, -nocommands, or -novariables are specified, then the corresponding substitutions are not performed. When it performs its substitutions, subst does not give any special treatment to double quotes or curly braces (except within command substitutions)

ex: set a 44; subst {xyz {$a}} => returns ``xyz {44}'', not ``xyz {$a}''

ex: set tmp1  [subst -nocommands {${var1}isprime/name/${var2}/A1}] => Here var1 and var2 are substituted by their values and var "tmp1" is set to the resulting string obtained

format: format cmd formats string in fashion similar to C sprint cmd using % conversion specifier.

  • %d => convert integer to signed decimal string,
  • %f => Convert number to signed decimal string of the form xx.yyy. If we have %5.2f, it means 2 decimal precision with minimum field width of number (including all the digits and the dot) to be 5. It will pad with spaces to make it 5 wide if the total width turns out to be less than 5. ex: format %6.2f 78.345 => prints " 78.34". Note the space added in the front to make it 6 field wide. If we want to pad with "0" instead of space, we should do %06.2f, then it outputs "078.34".

ex: format " Today, %d shares in %s were bought at $%.2f each" 123 "Global BigCorp" 19.372 => Prints: "Today, 123 shares in Global BigCorp were bought at $19.37 each". Here %0.2f means 2 decimal precision.

2. list: This is a composite data type. It's a group of elements, put within quotes " ... " or curly braces { ... }. list is an ordered collection of items. List can contain any kind of items, even other lists.

List created using any of below:

ex: set myVariable {red green blue} => assigns list to var. Could have used " " instead of {}

puts $myVariable => lists can be printed by providing list name, prints "red green blue"

ex: list a b => returns {a b} or "a b" => other way to create list is to use list cmd explicitly. So, set myvar [list a b] is same as set myvar "a b"

IMP: Be careful when using {} when making a list. If there are var, then it won't work, and we need to use "" if we want to subs those var. ex:

  • set tmp1 "aca"; set tmp2 "baa";
  • set mylist [list {a $tmp1} {b $tmp2}] => Here {a $tmp1} is assigned to index 0, {b $tmp2} to index 1. No subs done.
  • set mylist [list [list a $tmp1} {b $tmp2}] => Here {a aca} is assigned to index 0, {b baa} to index 1. Subs done. Correct way if subs is desired.

list operations: Many operations can be performed on lists, as sorting, getting nth element of list, etc. There are many list cmds available that can do that. list is most widely used dat type in Tcl, so knowing cmds that do operations on list is very helpful. Below are few imp ones:

lappend => appends to list

set var1 orange
lappend var1 " " "blue" => appends blue and empty var "" to var1. Empty var does nothing to var1. instead of lappend, we can use append.

llength => length of string (if 2 elements in string, this returns 2)

puts [llength $var] => gives length as 2

lindex {list} index => returns element from list corresponding to said index. list's first element is index 0.

puts [lindex $myVariable 2] => prints index[2] which is "blue"
puts $myVariable => prints all elements of list


lindex {a b c} 0 -> a
lindex {a b c} 2 -> c
lindex {a b c} end -> c
lindex {a b c} end-1 -> b
lindex {a b c} -> a b c (no index provided, so the whole list is returned)
lindex {a b c} {} -> a b c

lindex {a {b {c d} e} f {g h} i} => This list has lists within the list (nested list). Here index 0 is "a", index 1 is "b {c d} e", index 2 is "f", index 3 is "g h" and so on. to access sub-elements of index 1 , we need to assign index 1 to a var, and access each element using lindex for that list, i.e set tmp [lindex { .... } 1]; lindex $tmp 2 => returns index 2 of {b {c d} e} which is "e".
lindex {{a b c} {d e f} {g h i}} 2 1 -> h (2 dim list, so equiv to lindex (g h i} 1 => h

lassign => xform list to var:
set var {orange blue red green}
lassign $var colour1 colour2
puts $colour1 => color1 gets assigned orange, while color2 gets blue

lsort => sort a list. default is ASCII sorting, unless options -integer, -real, -dictionary is used. ASCII sorting is basically sorting based on ascii value of each character.
set var1 [lsort $var] => var1 is ASCII sorted list (in increasing order, i.e smallest items first). -increasing sorts in increasing order (default), while -decreasing sorts in decreasing order.

lsearch {list} search_term => lsearch returns the index of the first element in list that that matches pattern, or -1 if there are no matches. Before Tcl 8.5 introduced the in operator, lsearch was very frequently used to test for the existence of a value in a list.

ex: if {[lsearch $var att] > -1} {

...

}

3. Simple array: This is a another composite data type like list. Just like var, no need to define array. Here array indices are integers like 0,1,2, etc.

ex: array_name (array_index) => NOTE: round brackets used instead of square brackets (square brackets are used in C pgm). In Tcl, square brackets used for cmd substitution.

set color(0) purple

set color(1) red

puts "$color(0) $color(1)" => prints purple red. If we don't put "" then it errors out since puts can take only 1 arg, while here there are 2 args

puts [array size color] => "size" returns the size of array "color". Here it returns 2.

To read elements of array, we can iterate thru indices using for loop (in case they are continuous), loop ending when we reach the size of array.

4. Associative array => associative array is a more genric form (or a superset) of simple array, where array indices can be anything (doesn't have to be limited to numbers). All scripting languages support associative array as it's very helpful. It's in key(index) value pair. Index doesn't have to be a number like 0, 1, 2, etc as in simple array. When index is string, it's called associative array. index is number in C like language, but in tcl or other scripting languages that support associative arrays, it can be anything.
ex: set marks(english) 80
array set marks english 80 => same as above. NOTE: no ( .. ) when using array set.

array set val [list a 1 c 6 d 3] => multiple var set here, by having more than 1 pair. So, array val(a) set to 1, val(c) set to 6 and so on. list is optional

array set val {a 1 c 6 d 3} => equally valid form, same as above. NOTE: curly braces used instead of [ .. ] since list is a cmd that returns o/p.

array set val [list a [list clk [list c d] e] z [list x y]] => here array elements themselves are nested lists, so val(a) = "clk [c d] e" val(z) = "x y". NOTE: these are not treated as nested arrays or 2D arrays or anything, it's still 1D associative array, just that each element of array is nested list. We can access each sub-element of val(a) by using list cmd as "lindex" etc. i.e "lindex $val(a) 1" gives "c d"


puts $marks(english) => prints 80, puts $marks is an ERROR as all indexes of array can't be printed directly, each index has to called separately
puts [array size marks] => returns 1 as size of this array

set marks(student) "Dave" => here array marks stores "Dave" and not number. Each element of array can store anything (obvious as everything is a string)

puts [array get marks] => "array set" and "array get" are most popular for setting array and getting array. " array get" returns a list containing pairs of elements (index and value) of the array.

ex: we can get all array elements by using [array get ...] cmd above, but we can also traverse using foreach loop.

foreach {subject number} [array get marks] { puts "$subject -> $number" } => here we get index and value in each iteration of loop.

puts [array names marks] => "names" prints indices of array. here it prints "english" since that's the only index. This is used to iterate thru indices of associative array using for loop, compared to using "size" to iterate thru regular arrays. "array get" could also be used to get all elements of array directly or going thru a loop.

ex: foreach {idx_key} [array names marks] { puts "$marks($idx_key)" } => prints value of each element of array

puts [array names marks english] => here it prints the index english, if it exists in marks array. since $marks(english) exists, it prints "english". If english was not a valid index, it would have printed nothing.

ex: index can be hyphen or anything in combination of special char.

set results(-noexecute) false

set Me $results(-noexecute) => sets Me to "false"


operators:

arithmetic: + - * / %
relational: == != < > <= >=
logical: && || !
bitwise: & | ^ << >>
Ternary: ?: => If cond=true ? val=x : val=y
math func: abs, sin, cos, log, rand, etc
system func: clock, open, close, exec(execute system cmd)
ex: puts [exec ls] => executes ls cmd, and output is printed on screen

NOTE: there is no "=" operator in tcl as in C or other languages. To assign something as in tcl, we use "set" cmd explained next.


Cmds:

1. set => sets a vraiable. In most other languages "=" sign is used to assign a var, but in tcl, set is used instaed (there's no equiv "=" sign in tcl)
----
set a 5 => sets var a to 5. Use $a to access a. puts $a will print 5. here 5 is string type
set a "me" or set a {me} or set a me => sets var a to string me. NOTE: "set a me you" won't work as "me you" is 2 words and not interpreted as 1 string, unless put in quotes => i.e set a "me you"
set colorlist {red green} => list is group of words within {} or "". $colorlist lists all the elements
set colorlist1 [list red green] => same result as above, list is optional
set colorList3 [split "red_green_blue" _] => the cmd split, splits the colors and then list is set those 3 colors. => puts $colorList3 => red green blue

2.

3. expr: In tcl, we can't directly do any arithmetic, as everything (incl numbers) is treated as string, so string operation is done.Instead we use cmd "expr" or expression to indicate it's an arithmetic expression.

ex: set a 2; set b 5; set c $a+$b => sets c to 2+5 and NOT 7. To fix this we need sto do this: set c [expr $a+$b] => sets c to 7. 

4. exec:

5. regexp:

regular expressions: pattern match similar to perl can be done here. regexp is used everywhere in unix world. See the regular expression page. tcl regexp is consistent with linux regex standard. However, here regexp has to be inside { }. This prevents var expansion (i.e $a will be treated as literal $a without any expansion for var a). If you want expansion of var, then you need to use double quotes for pattern "pattern". Also regexp supported seems to be ERE and not BRE. So, () and {} are treated as metacharacters and NOT literals.

NOTE: Tclk 8.1 and above has moved to advanced regex, so the behaviour for tcl versions before and after 8.1 may be different. See this link for tcl regex: https://wiki.tcl-lang.org/page/Regular+Expressions

NOTE: When you move from {} to "" for patterns, make sure that additional escape chars are added. For e.g., if you had {\s*}, change it to "\\s*". I don't know the reason. Double quotes have unexplained behaviour, so I stick with {...} in patterns, as they follow what you would expect of regex behaviour.

  • ex:  regexp {.*\.\*$} $mytmp => This matches .* at the end of string. If we replace {..} with "..", then it matches anything as \.\* are interpreted as .*.*. However to match {...} behaviour, we have to put \\ before . and before *, i.e regexp ".*\\.\\*$" $mytmp => This matches the bahivour of {...} where .* is matched at the end of any string.

syntax: regexp optionalSwitches patterns searchString fullMatch subMatch1 ... subMatchn => patterns are searched for in searchString. Full match is any variable to hold the result of matched regex result. Submatch1 to SubMatchn are optional subMatch variable that holds the result of sub match patterns. These submatch happen if we have () inside the pattern, which implies that we want to hold the result of this match in a separate var.

return value of a regexp is 1 if matched, and 0 if it didn't match. So, this can be used in an if-else condition (NOTE: put [] around regexp to pass the return value)

  • regexp abc abcd => Simplest regex cmd, where "abc" is pattern searched in string "abcd". Here, it matches, so it returns 1.
    • if {[regexp abc ab]} { ... } => This doesn't match (as pattern abc isn't in string "ab"), so if part not executed.
  • regexp {([A-Z,a-z]*)} "Tcl Tutorial" a b => regexp is cmd that searches for string with any alphabets in the given string "TclTutorial". Any pattern matched string is assigned to var a. Since there is () inside the pattern, other match is also done in exactly same way, but the result is now stored in another var b. This returns a=Tcl, b=Tcl.
  • regexp {([A-Za-z]*.([A-Za-z]*))} "Tcl Tutorial" a b c => Here a is matched to the whole thing, with b also matched to whole string. c is matched to string within b as sub pattern (A-Za-z]*) is inside ([A-Za-z]*.([A-Za-z]*)). So, a=Tcl Tutorial, b=Tcl Tutorial, c=Tutorial
  • regexp {[0-9]+[ ]+:[ ]/} "12: 34 : 1234 : /My name" => this regexp matches 1 or more digits followed by 1 or more whitespace, then single : followed by 1 whitespace, ultimately followed by /. So here it matches "1234 : /". NOTE: whitespace match by giving a space inside square brackets. i.e: [space]. To match any of newline, tab or space do:  [\n\t ]+

switches: There are multiple switches or options that can be applied to regex via -. A "--" (double dash) implies end of switches.

  • -nocase: ignore case
  • -line: matching done only on a line, ignoring characters after a newline.
  • -start: Sets the offset of start of search pattern. So, -start 4 starts searching after 4th char (i.e 5th char onwards)
  • -- end of switches.

ex: regexp -nocase -line -- {([A-Z]*.([A-Z]*))} "Tcl \nTutorial" a b=> Here a and b, both match "Tcl" as newline prevents the regexp to go to next line (-line option)

regsub: Just like regexp is used for seraching matches, regsub can be used to substitute regex matches in a string.

Syntax: regsub opt_switches regexp subject replacement opt_resultvar => If opt_resultvar not specified then substituted string is apssed to the result of this cmd (which may be used to set another var, etc). -all switch matches and replaces all instances of patterns in the string. Without this switch only the first matching range is found and substituted.

  • ex: regsub -all {\mfoo\M} $mystr bar my_str => Replace (in the string in variable mystr) every instance of foo which is a word by itself with bar. Stores the final result in the same string my_str (NOTE: no $ in resultvar)
  • ex: regsub -all {.tcl} $my_list "" my_list => remove .tcl in $my_list var.

6. io cmds: cmds for input output of data. Just like scanf and printf in C. We use puts for printing, and gets for getting input. o/p cmd "echo" doesn't work in tcl, unless we do exec (i.e exec echo "abc"). However, echo does work when tcl files having "echo" cmds are sourced into CAD tools. I've many examples that use "echo" instead of "puts". That's not correct, and may not work. Use "puts" instead when working in tcl, as that's the proper print stmt.

  • A. puts: To output to screen or other files
    • puts -nonewline string => writes characters given by string on channelId which by default is stdout. puts normally outputs a newline character after string, but this feature may be suppressed by specifying the -nonewline switch.
      • ex: puts "my name is $name" => This prints "my name is Raj", assuming var "name" is set to Raj.
      • formatted printing: Similar to printf in C, we can formt o/p by using format cmd in tcl, which formats any string. See format cmd abobe in strings
        • ex: puts [format "%0.2f" $rate] => string is first formatted and then passed to puts cmd.
      • ex: Append a log message to a file:
        • set chan [open my.log a] => nested cmd open inside []. chan is set to filehandle returned by opening my.log in append mode (a=append, r=read, w=write)
        • set timestamp [clock format [clock seconds]]
        • puts $chan "$timestamp - Hello, World!" => since file is in append mode, this line is written at the end.
        • set file_data [read $chan] => reads file and stores contents in file_data. File has to be open in read mode.
        • puts $file_data => prints all contents of file_data on screen
        • gets $file_data data1 => gets data from file one line at time, and puts in var data1. use a loop to read whole file
        • close $chan
  • B. gets: To get input from screen or other files.
    • ex: Get a number from user and print it
      • puts "Enter a number
      • gets stdin my_var=> gets the input from std input (keyboard) and stores it into var named "my_var"
      • puts "Num entered is $my_var"
  • C. redirect => to redirect o/p of cmd to a file. We provide the name of the file (and NOT the pointer of file when doing "open file"). Reason is redirect will open and close the file for you, instead of you trying to open a file in wrt mode, write into the file and then close the file. This becomes very convenient as it's all done under the hood with redirect cmd. Do NOT use file pointer with redirect cmd. It will not write anything, nor will it show any error.
    • ex: redirect -file ~/tmp_file {[puts $marks(Mark)]} => -file specifies file_name to redirect the o/p to. Anything within { ... } is redirect to the file. "puts" is needed since it prints the o/p of cmd. Std unix redirection cmd ">" will not work for redirection.

8. flow ctl cmds: controls flow, via loops, conditional stmt, etc. Similar kind of keywords as in C: if-else, while, for, etc. Syntax for these is very stricyt in tcl, where starting/closing braes have to be in

A. if - else: You have to add if else start and end curly braces on same line (as shown below), except for the  final closing curly braces }. If we put starting or ending curly braces on separate line else it will error out with "unkown cmd {".
if {$x == 0} { => IMP: if starting curly braces should be on same line, exactly like this. You can't write if {$x == 0} on 1st line and "{" on next line, as then the cmd if won't be parsed correctly. gives error "wrong #of artgs" for if.
echo "Equal"
} elseif {$x > 0} { => IMP: write elseif exactly like this. ending "}" for if and starting "{" for elseif need to be on same line, so that elseif cmd can be recognized correctly
echo "Greater"
} else { => IMP: write else exactly like this. similar to elseif above
echo "Less"
} => This could also be put in previous line.

ex: if {$x == 0} {echo "Equal"} elseif {$x > 0} {echo "Greater"} else {echo "Less"} => This is same as above cmd, but everything on 1 line. This is equally valid, as all cmds can be parsed correctly here (since there are matching starting closing braces around keywords)

B :? => used to replace if else
set b [expr $a == 1 ? 20: 30] => assuming a=5, then, b is set to 30
#while
set p 0
while {$p <= 10} {
echo "$p squared is: [expr $p * $p]"; incr p
}

C. for: rewriting while loop shown above
for {set p 0} {$p <= 10} {incr p} {
echo "$p squared is: [expr $p * $p]"
}

D. foreach: implements a loop where the loop variable(s) take on values from one or more lists.
#simple list
foreach el [lsort [array names a]] { => each array name a is assigned to el
echo "a\($el\) = $a($el)"
}
#multiple items from list
foreach {name gender} [list Bob male Sarah female Kim hermaphrodite] {
... => name and gender assigned to "Bob and male" in 1st iter, then "Sarah and female" in 2nd iter, so on. $name and $gender can now be used inside the loop.
}
#multiple lists traversed simultaneously
foreach a [list 1 2 3 4] b [list 5 6 7 8] c [list a b c d] {
puts "$a $b $c" => "1,5,a" is printed in 1st pass, then "2,6,b" and so on.
}

E. break, continue: terminate a loop before the termination condition is reached, as follows:
The break command causes the innermost loop to terminate.
The continue command causes the current iteration of the innermost loop to terminate.

F. switch: equiv to if tree
switch $x {
a {incr t1}
b {incr t2}
c {incr t3}
}

9. tcl special cmds: These are special cmds, which are used very often inscripts, so I've listed few of them.
A. info script:
echo "Sourcing [ file normalize [ info script ] ]" => info script returns name of file containing the script that is currently being evaluated. file normalize file1 => returns normalized filename (no ../ etc) for name1. This echo line should be first line in any script to print name of file/script being run. For some reason, this gives an error with "echo is unknown cmd". Use puts [exec echo "..."] instead. Prints => Sourcing /home/kagrawal/scripts/hello.tcl

B. info exists:
[info exists ::env(VAR1)] => returns 1 if such env var VAR1 exists, and is defined, else returns 0.
set a 3;
info exists a => returns 1
info exists $a => returns 0, since $a is 3, so it looks for var named 3, which doesn't exist.

10. procedures: Similar to functions in C.

#procedures: reusable procedures can be written. proc name {args} {body}

var d; set glb_d "my name";
proc plus {a b} { variable c; upvar $a arg1; return [expr $a + $b] } => 1 line proc defn to add two numbers. expr=expression. Here variable "c" is only seen within plus namespace (see namespace section below). To see it globally outside the proc, there are 2 ways:

  1. global: Decalraing a var as global inside a proc, allows it to be seen outside the proc.Same "global" keyword is also used to make a var declared outside the proc to be visible inside the proc.
    • In above ex, use "global c" inside the "plus" proc. Then var c is accessible outside the "plus" proc
    • In above ex, use "global glb_d" inside the proc. Then var "glb_d" is accessible only inside the proc "plus". If we have "global glb_d" outside the proc, then it will have no effect, and won't be accessible inside the proc.
  2. upvar: upvar creates link to variable in a different stack frame. It simulates "pass by reference" seen in other languages as C. This is able to handle complex situations which global can't handle.
    • Syntax is upvar <level> <other_var> <my_var>. <my_var> can access <other_var> from corresponding stack frame. Default <level> is 1. We can provide <level> as number 0, 1,2, etc or #0, #1, #2, etc. When # used, the meaning changes slightly. "upvar #0" means at global level (i.e get values from top level). In above ex of proc, upvar makes arg1 an alias for var a, so that arg1 can be used inside proc w/o issues. Generally convenient to do it this way. Only use #0 or #1 (as per tcl doc) or you are looking for trouble.
    • In above ex: use "upvar #0 glb_d loc_d" inside the proc. Then "loc_d" var link is created for "glb_d". Now if we modify glb_d or loc_d, both var get modified as they are refrring to same mem location. Ex:if inside the proc, we do => set loc_d "you". then both glb_d and loc_d are set to "you"


puts [plus 5 6] => proc being called with args. returns 11. NOTE: no brackets should be used for args when calling proc, or else it will give "Error: wrong # args". Also proc called has to be put into [...] when it's being used as arg to other cmd. Else "plus" keyword will be treated as data, and will error out. if proc is used by itself on a line then we can write "plus 2 3" (without the quotes) and it will work fine. However, if we use [plus 2 3] by itself on a line, then we get [5] (with the sq brackets) which is not what we want.

 

define_proc_attributes => Defines attributes of a Tcl procedure,
ex: define_proc_attributes sta_flow -info "configure pt" -define_args { => -info "text" gets printed when help is typed for this proc, help -verbose prints all possible args below which is a list of lists. Can be just 1 arg too.
{-design "valid design name; NOTE: usually defined by project.globals" "" string optional} => this is arg1
{-stage "valid design stage; NOTE: usually defined by project.globals" "" string optional} => This is arg2 and so on ..
}

ex: factorial
proc factorial {number} {
if {$number <=1} {return 1}
else return [expr $number * [factorial [expr $number - 1]]]
}
puts "factorial of 5 is" factorial 5 => prints 120

ex: info body factorial => This returns the code in the proc "factorial". This is helpful during debug, so that we don't have to find where that proc is.

11. namespace: Namespace is a container for set of identifiers that is used to group variables and procedures. Namespaces are available from Tcl version 8.0. Before the introduction of the namespaces, there was single global scope. Now with namespaces, we have additional partitions of global scope.
Namespaces are created as below:

namespace eval MyMath { => creating namespace MyMath
variable myResult => var inside namespace, can be accessed via MyMath::myResult
proc Add {a b} { .... } => This proc Add is only within MyMath namespace
namespace eval name2 { ... variable myResult ...} => namespace can be nested too. ref would be ::Mymath::name2::myResult
}
proc MyMath::Add {a b } { => create proc "Add" inside MyMath namespace, need to explicitly use ::, since this proc is defined outside MyMath amespace { ... }
set ::MyMath::myResult [expr $a + $b] => assign MyMath var MyResult to result, :: refers to top level global namespace, and then further :: refer to lower level. We can omit global ::
}
MyMath::Add 10 23 => this sets MyMath::myResult to 33
puts $::MyMath::myResult => prints 33, we can omit global ::, i.e puts $MyMath::myResult

To avoid using lengthy scope resolution operator, we can import/export namespace
namespace eval MyMath {
variable myResult
namespace export Add => This says that make "Add" available to whoever imports MyMath. Using export/import allows names to be shortened
namespace export Add1 => export another proc Add1, do similarly for other proc that you want exported
}
proc MyMath::Add {a b } {
return [expr $a + $b]
}
namespace import MyMath::* => Here everything that has been exported, is imported in global namespace
puts [Add 10 30] => Since "Add" has been imported, it is available in global namespace, Var still available as $MyMath::myResult

12: packages: similar concept as in other pgm languages, where a group of unctions, header files, etc can be put in a set of files, which can be acccessed as a single entity.

creating packages => consists of collection of files (tcl scripts, binary, etc) that provide specific functionality. It has a pkg name. Once created, pkg can be called in other tcl pgms, by specifying path of pkg
To create a package with TCL, you need two files :
  1.  the package file which contain your code
  2. the index package file to declare your package

These are the stpes to create pkg and then use it in a pgm:

I. package file: create a tcl file called HelloPkg.tcl. This defines the pkg called "HelloP" with all procs and var in it. NOTE: name of pkg file (HelloPkg.tcl) can be different than name of pkg (HelloP) as well as the namespace "HelloWorld". Strictly speaking, namespace is not required, as we just load this file, so procs and var will be available to the whole pgm loading this file. However, convention is to have a namespace so that if multiple pkg are being loaded, and they happen to have same name of proc or var, then they will keep overwriting each other, creating errors in results. Also, we usually keep namespace and pkg name the same, so that it's easier to know from which pkg, the particular proc is coming from.

namespace eval ::HelloWorld {
        namespace export Talk => export proc Talk, so that it's available to anyone who uses "import HelloWorld::Talk"
        set version 1.0
        set HelloWorldDescription "HelloWorld"
        variable home [file join [pwd] [file dirname [info script]]] => Variable for the path of the script
}

proc ::HelloWorld::Talk {} { => define proc Talk
    puts $HelloWorld::HelloWorldDescription
}
proc ::HelloWorld::GetPath {} { => define proc GetPath
    variable home
        return $home
}
package provide HelloP $HelloWorld::version => "package provide" actually creates pkg. NOTE: we defined name of pkg as "HelloP" while namespace is "HelloWorld"

package require Tcl      8.0 => This is needed incase Tcl version is < 8 (then namespace won't work, as they are defined only for version 8 and higher)

II. index package file: This index file is simply a Tcl file which is loaded into the interpreter when Tcl searches for packages. This file is named pkgIndex.tcl.  pkgIndex.tcl can be written manually or can be autogenerated. We need to add this 1 line in pkgIndex.tcl:

package ifneeded HelloP 1.0 [list source [file join $dir HelloPkg.tcl]] => "package if needed <pkg_name> <version> <pkg_filename>" cmd loads this pkg (pkg_filename) in response to cmd "package require <pkg_name> <version>". pkgIndex.tcl file is sourced on application startup, but pkg files are not loaded at that time since "ifneeded" is used. If "if needed" is not used, then <pkg_filename> will be loaded while sourcing of pkgIndex.tcl, which may be unnecessary, as not all packages are used in every pgm. "ifneeded" prevents pkg from getting loaded. The second part of this cmd says what files to source when this pkg is going to be loaded. We need full path of pkg file for sourcing to happen. $dir is the path of dir where HelloPkg.tcl resides, so "file join" provides full path of file. NOTE: $dir is not a std var, and "echo $dir" returns "unknown var" error, but it works for some reason when used in pkgIndex.tcl.

package ifneeded HelloP 1.0 [list source [file join $dir hello1.tcl]]\n[list source [file join $dir hello2.tcl]]\n[list source [file join $dir hello3.tcl]] => We can load multiple files too for 1 pkg

To auto generate this file, we can cd to dir where our pkg file "HelloPkg.tcl" is located, open up tclsh, and run "pkg_mkIndex . *.tcl" cmd. This will generate " pkgIndex.tcl" file in that dir for that pkg. If there are multiple packages in that dir, then pkgIndex.tcl will have multiple lines, 1 for each package.

III. use package in external pgm:  Tcl searches for index pkg file in "auto_path" or "tcl_pkgPath" which store default paths to search.

echo $auto_path => /usr/local/lib /usr/share/tcl8.5 /usr/lib64/tcl8.5 /usr/lib64/tk8.5 /usr/share/tk8.5

echo $tcl_pkgPath => /usr/lib64/tcl8.5 /usr/share/tcl8.5 /usr/lib64/tk8.5 /usr/share/tk8.5

Since we may not want to put our local file in any of these standard /usr/ dir, we append any of these paths to add the path where our pkgIndex.tcl is going to be put. Now to use this pkg in any tcl pgm, add these 3 lines in the pgm, Hello.tcl

lappend auto_path "/home//scripts/test" => This is the name of dir where pkgIndex.tcl is located, we append this path to the search path
package require HelloP 1.0 => "package require" cmd loads pkg. here pkg "HelloP" version 1.0 is loaded.
puts [HelloWorld::Talk] => Once pkg loaded, all proc and var accessible via namesapce.

puts [HelloWorld::GetPath]

puts $HelloWorld:::HelloWorldDescription => To access a var in pkg, we need to precede it with $

On running the script, Hello.tcl, we get this o/p:

HelloWorld
/home/scripts/test
HelloWorld

12. Error cmds: Error handling in Tcl is provided with the help of error and catch commands. Basically it allows you to take a chosen action incase any cmd in script gives an error. This is useful for debug purpose in production.

syntax:

error message info code => message=error msg, info=error info stored in global var "errorInfo". code=error code stored in global var "errorCode"

catch script_or_cmd ErrorVar => script_or_cmd is any cmd that you normally execute in your tcl script, ErrorVar is the name of var used to store the "Error msg" that is provided in the error cmd above. Catch cmd returns 1 incase of error, else 0. Useful to use it within if stmt, so that we print some debug info incase of error.

Ex:

proc Div {a b} {
   if {$b == 0} {
      error "Error generated by error" "Info String for error" 401 => Error var stored in global
   } else {
      return [expr $a/$b]
   }
}
puts "Result = [Div 10 0]" => Here we are executing Div proc normally. Since we provided 0 as one of it's operand, it will print error msg from above
if {[catch {puts "Result = [Div 10 0]"} errmsg]} { => Here, we catch that errorinfo from above, and take a defined path on printing these errorvar that were stored above in proc. This is more user friendly and also easier to debug.
   puts "ErrorMsg: $errmsg"
   puts "ErrorCode: $errorCode"
   puts "ErrorInfo:\n$errorInfo\n"

}

catch "exec myscript $a 4b" => this will catch the error, bt is not doing anything with it. This also doesn't have a var to store error msg. This is not very helful use of catch. We can use catch cmd with all other cmds, but then script will be very long and unreadable. So, we should use catch only with cmds that seem likely to fail, like "opening a file", etc.

13.

 



Tk = ToolKit

Tk provides cross platform gui widgets. It can be used with other dynamic lang, not just Tcl

Hello.tcl
---
#!/usr/bin/wish => wish interpreter is needed to run Tk cmds, if we use /usr/bin/tclsh, thne it will not run as any Tk cmds are not identified by tclsh
grid [ttk::button .mybutton -text "Hello World"]
---

./hello.tcl => draws a box with "Hello World" in it. We can also type > wish hello.tcl => not needed since wish interpretor is already there on 1st line

special var:tk_version, tk_library

#!/usr/bin/wish
puts $tk_library => prints path of where all std Tk libraries are. Note: all tcl cmds are still valid in Tk

widgets: basic component pf Tk based app. aslo called window
-------
main widget is root widget, and other widgets can be placed in it. basic gui widgets are buttons, menus, etc.

create widget:
type variableName arguments options => type= widget type as button,label, etc. args are optional. options range from size to formatting of each component

ex: label .myLabel -background red -text "Hello World" => root window is named with . and an element in window is named .button1. button1 is var name. . refers that it's in root window

basic widgets:
-----------
label = Widget for displaying single line of text.
button = Widget that is clickable and triggers an action
entry => accepts single line of text as i/p
message = Widget for displaying multiple lines of text
text = Widget for displaying and optionally edit multiple lines of text.
toplevel = Window with all borders and decorations provided by the Window manager.

layout widgets => frame, place, pack, grid (widget to nest widgets packing in diff dirn)

selection widgets

media widgets

ex:
grid [label .myLabel -text "Label Widget" -textvariable labelText]
grid [text .myText -width 20 -height 5] .myText insert 1.0 "Text\nWidget\n"
grid [entry .myEntry -text "Entry Widget"]
grid [radiobutton .gender.maleBtn -text "Male" -variable gender -value "Male"
-command "set myLabel1 Male"] -row 1 -column 2
canvas .myCanvas -background red -width 200 -height 200
pack .myCanvas
.myCanvas create arc 10 10 50 50 -fill yellow

window maager => to handle top level window. . refers to main window
wm option window arguments

ex:
toplevel .t => creates top level window with name "t"
wm maxsize . 800 800 => max size of toplevel window is 800 by 800. Can't be extended by more than that by stretching with mouse
wm minsize . 300 300 => min size is 300 by 300
wm title . "Hello" => title displayed on window is "Hello"
wm attributes . -alpha ".90" =>
wm geometry . 300x200+100+150 => width=300, height=200, x position on screen=100, y position=150. This position determines where on your monitor screen the window is going to show up.
grid [button .mybutton -text "Hello \n World"] => places button on 1st line of grid


 

joomla:

Joomla is very popular open source Content Management Software (CMS) right behind Wordpress in popularity. It's little bit more complex than Wordpress, but has lot more options to customize your website. It's not written around making a blogging website, but to make a regular website. With wordpress too, you can make a regular website, but it will require more work. In joomla, you can start right from the box.

Joomla offical website is: https://www.joomla.org/

A lot of begineer's and advanced learning material is on: https://docs.joomla.org/Main_Page

A very good tutorial as always is on tutorialspoint website: https://www.tutorialspoint.com/joomla/

Joomla 3.9 is latest as of June, 2020. All material below is wrt version 3.9.

Joomla vs WordPress:

Before you install joomla, understand that there are some very basic things missing from joomla that are very standard in Wordpress. I hate joomla for that, but then the lure of having a "customized interface" in joomla made me switch (customization in wordpress is too much work, as it's mostly for blogging. However, there are too many templates available in wordpress, that can allow you to customize any website). Joomla has lots of advanced features, which most of the times are just more fancy and convenient, but don't really add any real value. In fact things of real value have been taken out of joomla, as I note some of them below.

1. Joomla does not have auto save feature. So, basically if you have been typing an article for an hour, and then suddenly realize you have to save, and then you click on save. But then something goes bad (internet connection lost, timeout, autologout etc). Now the page you were trying to save with latest content is lost forever. You still have access to older copy, but your work for last 1 hour is gone. It's not saved anywhere. Wordpress auto saves your articles every few minutes, so I've never lost any article there, but with joomla I've lost lots of articles. One option is to click on "toggle editor" button on bottom of editor, then do select all and copy (ctrl+a, ctrl+c), then paste (ctrl+v) everything in local editor on your local machine, and save it there. Then click on "save" button on joomla, and if for some reason, save doesn't succeed, you have a local copy, from where you can copy and paste again. NOTE: you should always save "html code", as saving raw text loses all markups as bold letters, underline, etc. Also, be default the session timeout is 5 minutes, which is too small and on doing save, it's guaranteed that it will log you out without saving your new content. I extended it to "couple of days", so now I don't lose my articles due to auto log out (better than nothing). This can be done by logging into administrator panel, and clcking on system->global configuration. Now select "system" tab on slight right on top, and scroll to bottom on "session lifetime". Set it to 15000 and save.

2. Joomla doesn't have any search feature to search for articles in your backend (administrator panel). In wordpress, you can search for anything in all articles in backend. This was very convenient, as many times I would not remember which article I put certain information in. Searching for this easily allowed me to find the article. In joomla, there is a search button in backend, but it only searches in title, which is not very helpful. For ex, if I'm looking for a term "TV show" in any of the articles, searching in backend will report only those articles that have this term in the title of the article. The only way to search is to make all articles "published", then goto frontend (i.e your website "xyz.com"), and then search over there. There it searches for that term in all articles, menu items, etc.

3. mouse right click doesn't work in editor when editing articles in joomla. So, you have to goto "Edit" button at top and click on copy,paste etc. This is not too bad, as there are also short cut keys that you can use (ctrl+c to copy, etc0

intsalling joomla:

I've joomla 3.9.10 installed. So, instructions below may differ for other joomla versions. Official joomla website is best resource for installing joomla: https://docs.joomla.org/J3.x:Installing_Joomla

Few tips:

  • Download Filezilla client to upload joomla files to your server (filezilla is ftp pgm to transfer files from 1 computer to other)
  • On some host, "localhost" doesn't work for hostname in database configuration. Get name of server from you web hosting provider. Ex: on ipage, it's <yourloginname>.ipagemysql.com (i.e if your login for ipage is matt2016, then hostname would be matt2016.ipagemysql.com), while on godaddy it's localhost.
  • Once successfully done, you may want to remove login panel from front page of your new website. Goto Extensions->Modules, then search for "login", and then where login form appears, "unpublish" it. That will get rid of the login panel.
  • To go to administrator site (aka backend of website), type your website name, and add "/administrator at end. ex: www.maaldaar.com/administrator/

joomla basics:

We'll log into adminstrator site (backend), and then look at various buttons on top. These are the icons you will see on top: System, Users, Menus, Content, Components, Extensions and Help. Most of the things under Users, Menus, Content and Extensions can also be accessed by clicking on System->Control panel.

Template: Templates are what determine how our website looks. It's a type of joomla extension, and you can download any joomla template on internet, and activate it on your website. That will completely change the look of your website. Thus templates make your website customizable. If we click on "Extensions-> templates" on top, we'll see 2 templates available by default (Beez3, protostar). We can also also see the 2 templates by going to system->control_panel->templates. Under styles, it will show all templates. Choose default icon for the template you want to have for your website (by clicking on default). protostar is default, but personally I prefer Beez3, so I change it to Beez3. All website creation notes below have been written with Beez3 in mind. The templates that we talked above are front end template. There are also 2 back end template that change the feel of the administrator panel (after you log in). You can see these by choosing "administrator" instead of "site" on left top side, on the screen where you see the 2 front end templates. The 2 back end templates available by default are Hathor and Isis. Isis is enabled by default. We rarely need to change it, so we never mess with back end template. 

Each template comprises a lot of files. Main file is index.php, which is called on loading your website. This in turn calls many other files which determine the look and feel of your site. You may change the code in the any of the files in this template, and that will change the website appearance. Thus templates contain the main code of your website, and they interact with contents of your website in order to display it. Files in a template can be viewed by by clicking on extensions->templates->templates (not template style), and then clicking on particular template details and files. You will see index.php as well as various dir. Whenever you want to change anything in template, you will need to mess with these files, so you will need to know some php, css and html coding.

3 important concepts to know regarding publishing a page or an article:


1. ARTICLE: first create article using article manager. Articles are the main content that you want displayed on your website (similar to wordpress article concept). We can see all articles by going to Content->Articles. When we see all articles list, you should make sure that you see a green tick mark in "status" in front of any article name. Else that article will not display on your website. You can add a new article by clicking on "new". By default, "content" tab is opened, which gives us a blank editor to type our content. On right side are multiple options: choose "status" as published, Category as "what you want" and other options. There are multiple tabs on top besides the "content" tab. "publishing tab" is a important one, which controls the"status" of the article. If the "finish Publishing" line has a date on it, then the article will expire on listed date, and will cause the "status" to go red. Delete the "finish publishing" line to keep article unexpired. On saving the article, you will see that each article gets assigned an "ID" number. This is the unique number that is used on the link to display the article. For ex: to see article with ID=23, http://www.maaldaar.com/index.php?option=com_content&view=article&id=23. However, when you go to that article on your website, it will have a different link like: http://www.maaldaar.com/index.php/linux/linux-installation. This is because joomla creates an alias for that link, which is human friendly.


2. MENU: However, articles by themselves are not visible on your website, unless they are linked via "menu". Menu is list of menu items. You can think of menu as "restaurant menu", which has various menu items as bread, burger, chips, etc. In joomla, menu items contain the final link to your articles. You can view menu as rectangular box with hierarchy of menu items in it. Usually, it's "top/bottom or left/right navigation gui" box, which has all menu items in it. Clicking on menu item takes you to the page, which has one or more articles in it. Infact, menu item is just a link to whatever you assign it to (we usually assign menu item link to an article, but it can be link to many other things. Joomla allows us to choose what we want that menu item link to point to). For ex: menu might be "Shopping", while menu items might be "electronics", "clothes", "jewelry", etc. All of these menu items together form menu. Do not think of menu and menu item as same thing, they are NOT. menu is a collection of menu items.

If we click on "menu" on top, we will see 2 menu provided by default: main menu and user menu. The default is main menu (shown by home sign). We can add menu items to "main menu" by going to meus->main menu->Add new menu item. On new screen, we choose "menu item type" to be "article", link it to an existing article, save and close. Thus the article gets linked to menu item. We can add more menu items, each of which has link to articles, etc. Ideally there should be a separate menu item for each article, if you want each menu item to point to a article. Any article that doesn't have a link via menu item, will never appear on your website. We can also add new "menu" on top of 2 default menu provided. Click on menus->manage->add new menu. We can have as many menu as we want. Usually, we put all menu items under default "main menu", and that suffices for simple websites.

So, what's the purpose of having more than 1 menu? It's so that we can have different custom menu to display on different pages or on different parts of the same page. For rx, we may have a side menu on our website on left side, which always shows all menu items. However, on top row of website, we may want to have a menu with different menu items as "About Us", "contact", "sitemap", etc. In such a case, we need 2 different menu. We'll see how to place these menu in module section.

We can also create a hierarchy of menu items, so that our website is easy to navigate. When we create or edit menu item, on right side, we see some options. One of the options is "Parent item". Here, by default, it's selected as "Menu item root", but can be chosen as any menu item already created. Then that menu item becomes parent, implying that this menu item is child of that menu item, and sits under it. Similarly, we can have more child menu items. This is very useful to keep your website hierarchical, instead of flat.

An example of such hierarchical menu item is a website that has "finance", "shopping", "real estate" as menu items, and then each of these have further menu items (i.e finance has menu items as banks, stocks, bonds, etc, while shopping has menu items as beauty, clothes, etc). We can link an article to each of these. If we want parent menu items as "finance" to contain something different (i.e have a link of menu items that are under it, or short summary of menu items under it, we can choose "archived articles" etc in it instead of "single article". So, from perspective of joomla, all menu items, whether parent or child, are treated same way. T

Other way to understand difference between menu and menu items is that the top level root menu item is called "menu", while all child items underneath it are called menu items. Thus even though menu is just the top level or parent of menu item, we differentiate b/w menu and menu item, since menu is the one that is called by modules (explained below).


3. MODULE: The 3rd important thing is module (after article and menu). We created menu above (with menu items in it), but it's not visible on website yet. Where and how to place it is decided by module. Menu will not show up on webpage unless it is linked to atleast 1 module (and that module is published or visible). This is because, modules are the ultimate thing that show up on webpage. In template "Beez3", we see html/modules.php file (by going to editor of beez3 template. It's in extensions->templates->templates and clicking on details and files). This file is called via index.php and all modules are loaded and rendered one by one. So, if some menu is not linked from within a module, it will not show up on the website. We can see all available modules by clicking on extensions->modules. For main menu above, we have a default main menu module. Similarly we have "login form" module, "latest articles" module, etc. Position of each of these modules (i.e where they are going to be placed on the webpage) is indicated under "positions". Position of these modules can be changed to anywhere on your webpage as left/right/top/bottom etc. For Beez3 template, main menu module is in position 1 (top) by default, while user menu module is in position 7 (left). Which position goes where on the page is determined by the template code, i.e in template beez3, developer of this template might define "footer" position as one on the bottom of the page, while position 7 might be defined as right of the page. So, when arranging positions of various modules, choose module position corresponding to that template.

We can create many more modules for many different positioning of same main menu. Or we can create new menu as "my custom menu", and create a new module for "my custom menu". Thus module allows us to have flexibility in placement of that box, which may be menu, search form, etc. Without module, menu would be at a fixed location, with same menu items being seen across all articles. The code is hardcoded into the template. But now, we can make many menu (each having different or similar menu items), then make many modules (with each module having different menu). Now on each webpage, we can decide what module to place, where to place, so that different menu will show up on each webpage. This is a big winning feature of having module in joomla (in wordpress, you have same menu across all pages. modules is a new concept introduced in joomla). Thus modules provides one extra level of customization.

As you can imagine, module can be of many type. We can define module as "menu type", where it contains menu. Or it can be "article type", where it contains articles. Thus we can create a new module, assign it a type, and then choose what to display when that module gets loaded (i.e display menu items, articles, etc). Mostly, we have modules as "menu type", as we create menu items to link to our articles. For this website, I already had a "main menu" module, which was of menu type, and it's position was assigned 7 (i.e left). So, when the template gets called, this "main menu" module gets loaded, which is linked to "main menu". So, each module is a rectangular box and it contains inside it whatever we want it o have. All these modules then get displayed on any page at positions assigned, and this is how our final page looks !!

Modules are confusing concept, and the positioning is even more confusing since each template may have it's own definition of what "position-0" means. To help with that, joomla allows positions of modules to be seen on your website itself. For that, first goto Extensions->Templates, then click on "options" icon on top right of that page. Over there, set "Preview module position" as enabled, save and close. Now open your webpage (www.xyz.com), and append ?tp=1 (tp=1 says enable "template positioning") to end of that link (i.e www.xyz.com/index.php/shopping?tp=1). This will show position and style (i.e Position: position-0,  style: none outline) of all the modules that are rendered on that webpage. This is very helpful in understanding and debugging module positioning related issues.

Summary: So, to get your first webpage to be published, you need to understand all 4 things -> article, menu, module and template. Write a new article, then create menu/sub-menu which ultimately have links to articles. Then create or use existing modules to control placement and appearance of these menu/sub-menu, search, tags, etc. Templates are ultimate top level website files, which call and render these modules. You do not need to put any extra code in your template, as modules are rendered by joomla code itself, w/o any special code in your template.

website creation:

That is all we need to know for website creation. Remember that for building a simple website, we need to know very few (< 10) buttons of joomla. We'll just learn about the basic ones now, and then build on top of it later.

1. Log into joomla backend. On the very top, you will see 7 pull down menu:

  • system => 2 important links here are: "Control Panel" and "Global configuration".
    • CONTROL PANEL=> It's most important one, as it has links to articles, menus, modules and templates which is all we need to get a working website. In addition to these, it has links to "media", "categories", "user" and Global configuration settings. So, basically we do not even need to go to any of the next 6 pull down menu, as everything of interest is listed under "Control panel".
    • GLOBAL CONFIGURATION => This has global settings for various things on our website, and can also be accessed via "control panel" above. All default settings are good for us, no need to change anything here. Here we globally set options for everything as articles, modules, etc. This page also shows up when we click on settings on left top of page (only seen when we are editing something).
  • users => I just have 1 user for my website (which is administrator or me), so no need to mess with anything here. This is irrelevant for most single adminitrator websites.
  • menus => This is to manage menus and menu items. We use this to create any new menu or menu item. "Main menu" is already provided by default, so we can add all menu items to this "main menu". Add menu items as described above. We can add new menu also, so that we can other menu besides "main menu". This helps us to keep different menu items under different menu and show them differently by showing different menu on different pages.
  • content => We use this to add any new articles by clicking on articles. We can also upload files, images, etc by clicking on "media". Add articles as described above.
    • MEDIA: We can upload upto 10MB size of file in "media" section. You can change default of 10MB to any size by clicking on options on right, or going to system->configuration. The files can be pdf, jpeg, or any kind of video or text file. The path to this folder is "your-website-name/images". So, for a folder in "Media" called "sample" with file abc.pdf in it, the path would be www.mysite.com/images/sample/abc.pdf. We can also specify url as relative to main server path, so url as "images/sample/abc.pdf" is also recognized as a valid url.
    • UPLOAD FILES: The upload feature doesn't allow us to upload any html files, scripts/programs, or any other file that joomla thinks are unsafe. In order to upload such files, you will need to login to your website provider admin page, and directly upload those files in "images" folder. The dir for media is in your joomla_installation_dir/public_html/images.  So, for abc.com, it's in domains/abc.com/public_html/images. Goto this dir. You will see "index.html" file in every folder over here. You can create your folders here, though that's not preferred since then you will miss index.html file here. Best option is to create all your folders from joomla admin page using "create folder", and then upload files using this web server login page (and using file manager). Goto appropriate subdir in images, and upload files there. Or you can use ftp to transfer bunch of files in 1 go.
    • TEXT/IMAGE LINK: For putting link as a text for any article, we can choose Insert->Link drop down menu, and put the URL and the text and other details, and that produces a "text" link. To put a link as an image (instead of a text), first insert an image in an article. Once image is there, select image and click on "insert link". Now you can put the URL that you want your image to point to. If you toggle editor, you should see code like this: <a href=...> <img src=...> </a>. Or you can edit html code directly by going to html editor. This image serves as a thumbnail now. If you want to change shape/size of this thumbnail image, highlight the image, and click on edit image. Now you can edit this image just like any other image in your article. change Dimensions to 800 X 200 or something. Select "constrain proportions" if you want to keep original ratio of width X height of photo.
      • This image link is very helpful when you have a big image to display in your article, but you don't want that to take the whole page. Instead you want to show a scaled down copy of the image (i.e a thumbnail), and then when the user clicks on that thumbnail, it pops up the full size image. This looks neat and is very commonly used on websites. In such cases, the thumbnail image and the full image both have same url, and are pointing to same image on your joomla server.
    • LINK TO SECTION OF PAGE: One other kind of link you may want to put is a link to a particular section of a page. This can be done via putting "#" in href link and putting the "id" in the linked page. Again, use the html code editor directly to put the "id" in the section of page you are linking to. See in html section for details of syntax.
    • MODIFY HTML: Most of the changes you want to do in any article for which you don't have a button available is often easier to do by directly modifying the html code in the editor. Do not install any plugins, extensions or anything else for this.
  • components => we do not have anything of immediate need under this, so we can safely omit this whole section.
  • extensions => This has links for adding/updating extensions, modules and templates. Since we are going with template beez3
    • modules => This shows all modules, i.e main menu module, login module, search module. You can change the status of module to "unpublish" by clicking the status for that module. You can also add a new module by clicking on new module. We have to select a module type, and then fill in details. If we select module type as "menu", then on next page, we enter the menu that we want in this module, starting/ending levels, etc.
    • plugins => plugin is a kind of joomla extension. You can write your plugin to add new functionality. There are lots of useful plugins installed by default.
    • template => This allows us to see all available templates. If we haven't added new templates (by clicking on extensions->manage->install), we'll see 2 default templates provided by joomla. If we click on any template and click on "advanced", we can change template image, title, etc. Very important to change logo, title and description to match your website.
  • help => This has useful help links. Not needed for now.

2. Now, we add a new article using "content" pull down. After saving it, we add a new menu item using "menus" pull down. We add a new menu item under "main menu". We do not add a new menu yet, as we want to stick with default "main menu" provided. We just add a menu item under main menu. Once menu item is added, we now goto extensions pulldown and click on module. We see about 8-9 modules. We click on "main menu" module and make sure that selected menu on this is shown as "main menu". Next we click on extensions->template and make sure the right template is chosen.

3. Once article, menu, module and template are done in step 2 above, our website is ready to show our first article. Refresh your frond end page, and see if it shows your menu item and your article linked to that.

4. Under global configuration->site settings, we should change the name of site to our site. Remember to save any changes that you make, otherwise the change doesn't take effect. That's true for everything.

Helpful joomla settings:

1. Show menu at only 1 level of hierarchy at a time: One of the things that I've wanted to do is to hide sub menu on the main page, and only show the top level menu items. If the user clicks on the menu items, only then it should show lower level menu items. That way your menu items on main page don't show like 1000's of menu items at the same item, which will make it hard to read. One setting to achieve this is via:

 Extensions->Modules. Then select "main menu". It wil take you to "main menu" with 4 tabs, by default, module tab will be selected. On it, click on "show sub menu items" as "No" instead of default "yes". Do not change the start and end level, else sub menu items may not show at all beyond the end level, even on clicking on menu items on the main webpage (which is not what we want). "Click "save & close".

2. Increase timeout duration: set session timeout to 15000 as explained on top of this article. That will prevent a lot of your unsaved work from getting lost.

3. Reordering menu items: For menu items, you will notice that default ordering is decided by the order of creating the menu items. So, if you have menu item "physics", and you wanted to create sub menu items "chapter 1", "chapter 2" and so on, but if you created "chapter 2", before "chapter 1", then chapter 2 will show above chapter 1 in menu list. To fix this, we can change ordering of menu items at any time. This ordering is only available once you have already saved the menu item. After saving, you can click on "ordering" on right side, and click on menu item, after whch you want tyour current menu item to be placed. So, for ex, if you want to place chapter 2 and after chapter 1, then click on chapter 1, and save. Now, on the "ordering" pull down menu, you will see correct ordering. You can also check for correct ordering by refreshing your website page.

4 Separator between menu items: Sometimes, we want a separator between menu items, or something with no link, but just a bold text which separates one submenu from another. This allows for easy navigation. One way to do this is to create a new menu item, but instead of selecting that "menu item type" as "aricles", we select it as "system links". Here we can select it as "menu heading" or "separator" (both seem to do the same thing, but "separator" doesn't change text color/size as explained below. Maybe it has different class. So, stick with "menu heading"), and save it. After saving, we can put it in the correct order by selecting "ordering" on the right side and saving it again (as explained above).

The separator created above has same font size + color as other menu items, so it doesn't look like a real separator. To change the CSS properties like color, font-size, etc we need to edit the user.css file. Here's a link on how to do it: https://joomla51.com/tutorials/template-customization/applying-color-to-a-single-menu-item-in-joomla

In short, goto extensions->templates. goto "Beez3 details and files" or files of whatever is your current template. It takes you to the dir of that template. goto "css" folder, create a new file called "user.css", and write your custom css code here (seems like user.css is already provided in latest joomla version, so just add your code in there). This will overwrite other css proporties provided by default. We can make changes directly in any of the css files also, but then any changes you make to any of these files will be lost when you import new version of joomla (as beez3 and protostar templates are provided with joomla package and as such get overridden). user.css does not get over written by new installation, so any css changes to any file should be made in the user.css. user.css is the last css file to be loaded when joomla is running, so any settings in css/user.css will over write css settings in any other css file. So, don't change any other css file in template, just copy the code from other css file to user.css and make required changes in user.css.

In the ex provided in link, "item-210" is the class selector for menu items, where 210 is the id of that menu item as seen under that menu item listing.

If we want to apply a particular set of properties to all elements of a certain class, the better way would be to first find the class of such elements, and then apply the property. To find the class, we can look at html code of any page of our website. If I go to firefox browser, bring up my website and look up the source code, and search for any of menu item separator item (i.e finance), I see that the class for these is "nav-header", so I apply the properties to this class as below. In my case, I coded as below to change color to red and font-size to large, with padding to provide some extra separation b/w menu items and menu separator. 

.nav-header {
  color: red;
  font-size: large;
  padding: 15px 0px 10px 0px; /* provides top padding of 15px, while bottom padding of 10px. Top padding doesn't seem to work?? FIXME ??
}

5. Delete menu items and articles: When you trash a menu, menu item, article, etc, they don't get deleted, and it's possible to retrieve it from trash. However, sometimes when you try to create something with same name, joomla will not allow, even if the item is in trash. In such cases, you will need to delete that trash item permanently. The way to do it is to go to "search tools", when you are at at "all menu items", "all articles", etc, and then choose "status" as "trashed". then it will show "all trashed items". then click on "empty trash", and then item will be deleted permanently.

6. Featured articles: By default your first menu "Home" is shown on front page. Whatever is in the "home" article is what is displayed. However, you can make joomla show certain featured articles on this page, so that it looks more lively and recent, then having same stale content for ever. A very good link on how to do this:

https://docs.joomla.org/Managing_the_Featured_Items

In short, these are the steps. Change the menu item type of  "Home" menu item from "single article" to "featured article" by going to articles->featured articles. You can customize the payout of this page by changing settings under layout tab. You can also set the article order here. Now save "Home" menu item. At this point, you can go to articles page, and designate those articles as "Featured" which you want to display on "Home" page, and they will show up. You can also have a separate "Article" menu to show all articles under that menu. Then you can have a subset of these articles marked as "featured" and show them under "Home" menu.

One of the common things when displaying featured items is to show only a small introductory section of article, and not the whole article. That allows readers to see a lot of articles at once and they can click on articles of their choice. One way to achieve this is to add a "read more" by going to article and clicking "Read More" (3rd or last line on the article tab where you select size, link, etc for article). This adds a "red line" on your article content (in html code, you will see <hr id="system-readmore" /> at that place). However, when you go to your website, and look at this featured article, you won't see any text associated with this "Read More" tag. By default, "Read More" text is hidden. To fix this, goto menu for "Home" (or whichever menu is assigned for featured articles). There goto "options" tab, change "show read more" to "show" and "show title with read more" to hide (this may be needed only for beez3 template, since that template adds a title to be displayed by default on any read more button). You can change any other settings here too (like showing the article publish date, number of hits, etc). You can add a "introductory image" to an article, and that will show up alongside the article in featured article list.

One other way to show articles of certain type only is to put them into a ceratin category, and then create a menu with menu type as "articles->category list", and then under "choose a category", enter that category. then all articles belonging to that category will show up.

Writing new template:

Beez3 template:

We used the default "beez3" template above, but it's good to know how to write our own template. Before we write our own template, lets look at all the files on beez3 template. To do this, we goto extensions->templates->templates and click on details and files. It shows us few directories and bunch of files. We can also goto our webserver, and look in this folder to see these files:

website_server/public_html/templates/beez3/ => In your case the website server name may be something like abc.com

A short description of these files is provided on joomla website here:

https://docs.joomla.org/Understanding_Joomla!_templates

Important Folders:

css => lots of css files

html => lots of dir within it. contains template override files

images, javascript, languauge => these dir are not so important

 Important files:

templateDetails.xml => holds meta info related to the template

index.php => It's the main skeleton of webiste, and is the main file called for any page that is shown on the website. Fe details of this file:

  • Adds all the CSS stylesheets which are in css dir as layout.css, position.css, general.css. If user.css file is present, it's loaded at the end, so that it overrides all other css files
  • Main html file starts after that:

component.php, error.php => not so important for now

Few customizations to beez3 template:

1. Removing fontsize buttons: In beez3 template, by default, there is a "font size = Bigger Reset Smaller"  section on top right corner (just before the search box), which looks annoying to many. To remove this, we can just comment out this section in index.php. Just comment out line 130, which is "<div id="fontsize"></div> " as shown below (NOTE: it's html code in this line, so comment has to be in html syntax)

<!-- <div id="fontsize"></div> -->

2. Removing banner: In Beez3 template, you will notice a big banner with "joomla logo on left and and We are Volunteers message on right". The easiest way to remove this is to modify index.php file.

<div class="logoheader">  ..... </div> => This is the section that is responsible for putting that banner. Comment out the whole section as follows

<!-- <div class="logoheader">  ..... </div> -->

However modifying this file as above got rid of the container itself, which causes your top menu items (that can't fit the width of browser) to disappear when reducing the width of browser. The div container earlier used to hold those roll over of menu items from 1st line to 2nd line. so, instead we'll just remove the logo section and then take care of that div container other way.

Comment out the logo section as follows. (This preserves the div container as is. Don't comment out the div section, just the <h1> content inside it):

<!-- <h1 id="logo"> .... </span></h1> -->

 That gets rid of the logo, but the background picture is still there, as the css code for class="logoheader" is still present in personal.css. Here background image is set to "personal2.png" which is what we see on our webpage. This picture is not in media folder of joomla, but in "images" folder of beez3 template.

.logoheader {
    background: url(../images/personal/personal2.png) no-repeat right
     bottom #0C1A3E;
    color: #FFFFFF;
    min-height: 250px;
}

Instead of modifying the code here, we copy it in user.css, and modify it over there. That way it overrides whatever is in personal.css with the new settings in user.css. We should never modify any css file except for user.css (for reasons already mentioned)

In user.css, modify the background image to some other image, or just set background image to none.

.logoheader {
    background: url(../images/personal/bg2.png) repeat right bottom #0C1A3E; //here we set it to some other image already present in that folder with repeat option so that the smaller sized image repeats itself. NOTE: bottom #0C1A3E is part of "background" and should have been on same line for clarity (see ;  in end)   
    color: #FFFFFF;
    min-height: 250px;
}

If we don't want to put a picture at all, and just put some color or leave blank, we can set that as follows:

.logoheader {

    background-image: linear-gradient(red, yellow, green); //Use this option if you want to have some colorful design there. Many more functions available in CSS documentation
    background-image: none; //Use this option if you want no image at all
    color: #FFFFFF;
    min-height: 50px; //Change the height to 50px, since we don't want a big blank section in there. If we set it to 20px or less than only 1 row of "menu items" can fix, which causes menu items to dis appear if width of browser reduced too much. 50px allows you to have 2 rows, so it's somewhat better as it allows more menu items to show until they disappear
}

3. changing width of page: By default, you will see that your joomla website doesn't extend all the way to the right side, i.e if you keep on increasing the width of your browser, the website stops growing in width on right/left side, and will instead start showing blank on right/left most side. We'll need to change width attribute, and the place to change it is in user.css of the beez3 template (as explained under item 4 above in joomla settings). A sample code to change width is as below:

/* copied from file css/position.css, changing max-width from 1050px so that 15 inch laptop screen can be full width */
#all {
    margin: 0 auto;
    max-width: 1550px;
    padding: 0;
    text-align: left;
    font-size: 0.8em
}

/* copied from file position.css, above code didn't change article width, just page width. This changes article max width from 660px */
#wrapper2 .item-page {
    max-width: 1260px
}

/* copied from file position.css, changed width from 21% and added max-width to keep navigation menu on left from getting too wide */
.left1 {
    padding: 0;
    float: left;
    margin: 10px 3% 10px 0;
    width: 16%;
    max-width: 80px
    position: relative
}

4. Adding footer: Beez3 template doesn't show footer on webpage by default. However it's really easy to add footer via 2 ways: First is where we use the footer module that is already provided. We click on "Modules", then new which shows all modules available. We choose "footer" module and place it in position 14, which will place the footer on bottom of page. Second way of adding footer by choosing "Custom" when clicking on new module. This custom footer module can be written by putting in whatever content you want in the editor of footer module. If you want the footer content to be centered, then you can put this html code in Editor:

<p style="text-align: center;">  Copyright © 2020 Maaldaar </p>

One thing to note is that since we extended the width of our website, we should increase the width of footer module too by adding this in user.css:

#footer {
  max-width: 1550px
}

A very good video for this is here: https://www.youtube.com/watch?v=orWoxsjD170

5. Changing default joomla icon to your own website icon: Beez3 template (or any template) by default shows joomla icon on the tab where you open website. It's called "favicon" (favorite icon). You would want to change it to some other custom logo or picture. You can create or find an image for your logo, and then goto https://favicon.io/, and then use the converter over there to create a ico file from png or jpg file. Once you have created ico file and downloaded it in your local computer, you now goto your webserver hosting panel to upload this file. You can't use joomla admin interface of your website to upload this file, as it doesn't support uploading these kind of files. Once you goto dir: templates->beez3 of your server website dir, you need to upload this file with the name favicon.ico in this dir. any other name won't work. Now if you refresh your website multiple times, you should see your new logo on tab of your browser.

More details here: https://docs.joomla.org/Special:MyLanguage/Changing_the_site_favicon

6. changing page title to be displayed on browser for any menu item: By default, page title shows the menu name. Sometimes we may want to change this default behaviour. For ex: menu item "home" (which is the homepage for yur page) shows "home" by default on the title of the page. We may want to change it to the name of the website. For that, go to menu item "home", then click on tab for "page display" and change the title to "maaldaar" (or in your case your website name). Now whenever you goto your website, the default title shown on browser tab will be "your website name". NOTE: "home" title still remains for the menu item.

7. 404 Error page: By default, 404 error page shows joomla error page, with all joomla content. To customize it, you need to create a custom 404 page, and then link to it in error.php file. There is a good article about this: https://www.joomlashack.com/blog/joomla/404-pages/

First create an article with any title for your 404 page, then create a menu item for that article. "Unpublish" it so that it doesn't show up in "menu list". Now goto "error.php" file in Extensions->Templates->Templates->Beez3_details_and_files. Over there enter the line as below just after "defined('_JEXEC') or die;"

 if ($this->error->getCode() == '404') { header('Location: /theURLtoYour404Page'); exit; }

Leave rest of the code unchanged. You can remove it too, but it doesn't matter, as we already did a exit in the line above. So, rest of the code never gets executed. To remove the top bar from our error page (" the box that shows "Published, Hits, last updated, etc), we have to hide all those from showing by changing settings for this article on artcle page or menu settings.

New template from scratch:

The best way to learn templates is by writing your own template. This link explains it very nicely:

https://docs.joomla.org/Creating_a_basic_Joomla!_template

Once you have written your template, set your default template to your own template, and test it on your own website. Even though your website won't look fancy, but it will work !! I haven't gotten time to write my own template yet, but someday I will.