Files, directories and external programs

Every EDA run leaves files behind: logs, timing reports, netlists, checkpoints. A script that can open them, find them, write new ones and call other programs can automate the work between tool runs. This lesson covers those commands and ends with a small tool that summarises a log by message id.

Reading time: about 9 minutes.

9.1 Writing a file

open returns a channel name. You write with puts, and you must close the channel when done.

set fh [open notes.txt w]
puts $fh "line one"
puts $fh "line two"
close $fh

The second argument is the mode: w creates or overwrites, a appends, r reads. Forgetting close can leave the last lines unwritten, so pair every open with a close. Lesson 6 showed that try ... finally guarantees the close even if an error occurs.

9.2 Reading a file

There are two styles. read takes the whole file into one string. gets reads one line at a time.

set fh [open notes.txt r]
set data [read $fh]
close $fh
puts [string length $data]
puts [llength [split [string trimright $data] \n]]

set fh [open notes.txt r]
while {[gets $fh line] >= 0} { puts "got: $line" }
close $fh

This prints:

18
2
got: line one
got: line two

gets returns the length of the line, or -1 at the end of the file, so >= 0 is the correct loop test. An empty line gives 0, which is why the test is not simply "true". For a large log, read line by line. A multi-gigabyte file in one read can exhaust memory.

When a file cannot be opened, open raises an error. Catch it to give a clear message:

puts [catch {open nosuch.txt r} err]
puts $err

This prints 1 and then couldn't open "nosuch.txt": no such file or directory.

9.3 The file command

file is a family of subcommands for names and the file system.

set fh [open notes.txt a]; puts $fh "line three"; close $fh
puts [file exists notes.txt]
puts [file size notes.txt]
puts [file extension notes.txt]
puts [file rootname notes.txt]
puts [file tail /a/b/c.rpt]
puts [file dirname /a/b/c.rpt]
puts [file join /a b c.rpt]
file mkdir out/rpt
puts [file isdirectory out/rpt]
file copy notes.txt out/rpt/notes.bak

This prints:

1
29
.txt
notes
c.rpt
/a/b
/a/b/c.rpt
1

Build paths with file join rather than gluing strings with slashes, and take names apart with file tail, file dirname, file rootname and file extension. file mkdir creates all missing directories in the path. Other subcommands include file rename, file delete, file mtime and file normalize.

9.4 Finding files with glob

glob returns the names that match a pattern, in the same glob style as string match.

puts [glob -directory out/rpt *]
foreach n {a b} { set f [open $n.tarpt w]; puts $f x; close $f }
puts [lsort [glob *.tarpt]]
puts [catch {glob *.zzz} err]
puts $err
puts [glob -nocomplain *.zzz]

This prints:

out/rpt/notes.bak
a.tarpt b.tarpt
1
no files matched glob pattern "*.zzz"

glob raises an error when nothing matches, and the last line is empty because -nocomplain returns an empty list instead. Use -nocomplain when "no files" is a normal case, and sort the result, because the order is not defined.

9.5 Running other programs with exec

exec runs a program and returns what it wrote to standard output.

puts [exec echo hello]
puts [catch {exec false} err opts]
puts $err
puts [lrange [dict get $opts -errorcode] 0 0]

This prints:

hello
1
child process exited abnormally
CHILDSTATUS

Two rules. First, a program that exits with a non-zero status makes exec raise an error, so wrap it in catch. The -errorcode holds CHILDSTATUS, the process id and the exit status, which tells the caller which. Second, any text a program writes to standard error also turns into an error unless you redirect it, so exec cmd 2>@1 or exec cmd 2>/dev/null is common.

9.6 Environment and arguments

Environment variables are in the array env. A script started from a shell receives its arguments in argv, with their number in argc.

puts [info exists env(HOME)]

This prints 1. A script that takes a file name checks argc first:

if {$argc != 1} { puts stderr "usage: count_messages.tcl logfile"; exit 2 }
set logfile [lindex $argv 0]

Run as tclsh count_messages.tcl with no argument, it prints the usage line and exits with status 2. The same variables are available when a script is run from a tool's batch mode, which Lesson 11 covers.

9.7 Counting messages in a log

Let us consider a log in which each message line starts with a severity and a message id in brackets. The file below is synthetic, written to show the pattern. The message ids come from the Innovus reference, but the lines are not real tool output.

Innovus synthetic log (not real tool output)
**WARN: (ENCTS-3): clock net skew group not found, using default
**ERROR: (ENCTS-423): no library found for cell u1
**WARN: (NRIG-1323): net n_42 has no route guide
**ERROR: (ENCTS-423): no library found for cell u7
**WARN: (CHKCTS-16): sink pin ff_a/CK has no clock arrival
**WARN: (ENCTS-3): clock net skew group not found, using default
Finished step place
**WARN: (NRIG-1323): net n_77 has no route guide
**WARN: (ENCTS-3): clock net skew group not found, using default

The tool combines three ideas from earlier lessons: a regexp with capture groups (Lesson 5), dict incr for a tally (Lesson 8), and a sort of pairs.

if {$argc != 1} { puts stderr "usage: count_messages.tcl logfile"; exit 2 }
set fh [open [lindex $argv 0] r]
set counts [dict create]
set sev_total [dict create]
while {[gets $fh line] >= 0} {
    if {[regexp {^\*\*(ERROR|WARN): \(([A-Z]+-[0-9]+)\)} $line -> sev id]} {
        dict incr counts "$sev $id"
        dict incr sev_total $sev
    }
}
close $fh
set rows {}
dict for {k v} $counts { lappend rows [list $k $v] }
foreach r [lsort -integer -decreasing -index 1 $rows] {
    lassign $r k v
    puts [format "%-18s %3d" $k $v]
}
dict for {sev n} $sev_total { puts "total $sev: $n" }

Run with tclsh count_messages.tcl sample.log, it prints:

WARN ENCTS-3         3
ERROR ENCTS-423      2
WARN NRIG-1323       2
WARN CHKCTS-16       1
total WARN: 6
total ERROR: 2

The lines that do not start with a severity are ignored, so the header and "Finished step place" cost nothing. The most frequent warning comes first, which is the one worth looking at before anything else. This is the shape of a log reviewer. The exercise in Appendix B asks you to extend it.

9.8 A small Innovus example

Innovus can send command output to a file with redirect, and source runs a script file. The reference lists redirect timing.txt "timeDesign -postRoute" -append -tee as a pattern, and source takes -quiet, -verbose or -echo. To be run at the Innovus prompt. Check the options with redirect -help and source -help.

Legacy UI.

redirect timing.txt "timeDesign -postRoute" -append -tee
source -echo my_flow.tcl

The first line writes the output of the timing step both to the screen and to the file timing.txt. The second runs a script and echoes each command to the log, which makes a long flow readable afterwards.

The timing step also writes its reports to a directory. The reference states that timeDesign writes per-group reports with the extension .tarpt into ./timingReports by default, and that -outDir and -prefix change the directory and file-name prefix. A script can then collect them with glob:

foreach rpt [lsort [glob -nocomplain timingReports/*.tarpt]] {
    puts "report: [file tail $rpt]"
}

The -nocomplain keeps the script quiet if the directory is empty, and file tail strips the directory from the name.

9.9 Why an interviewer asks about this

Typical questions: "how do you read a file line by line", "how do you run a shell command from Tcl and get its status" and "write a script that counts the errors in a log". Good answers use gets in a while loop, catch {exec ...} with -errorcode, and regexp plus dict incr. A follow-up may ask what happens if the file does not exist, and the answer is that open raises an error, which you catch.