debugging.md 9.12 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
Debugging with TaPaSCo
======================

TaPaSCo designs don't have bugs, so no debugging facilities are required.
Right? Right. Well.

...

You're still reading? Goodness, why? ;-) Ok. so let's assume hypothetically that
your design would have a bug as a Gedankenexperiment. This document is about the
available facilities in TaPaSCo to hunt it down.

Table of Contents
-----------------

  1.  [Hunting hardware bugs](#hw-bugs)
      a.  [The debug feature](#debug-feature)
      b.  [Exercise an ILA](#use-ila)
  2.  [Hunting software bugs](#sw-bugs)
      a.  [tapasco-debug](#tapasco-debug)
      b.  [tapasco-benchmark](#tapasco-benchmark)

Hunting hardware bugs <a name="hw-bugs"/>
---------------------

To test the hardware interface of PE modules, use the peek and poke utilities in
[tapasco-debug](#tapasco-debug). It allows you to manually interact with the
registers of the module, peeking at the ISR and other registers, and even start
jobs.

If the bug does not concern the PE module interface, but something else in the
design, it is best to instantiate a (System) Integrated Logic Analyzer (ILA)
core in the design: ILA can record small traces in on-chip buffers, which can be
read out via Vivado to see actual signals in the running hardware. Recording can
be triggered by simple and complex conditions, making it feasible to debug on
real hardware. Since this is something that one needs to time and time again,
there's a TaPaSCo _feature_ that can automatically instantiate ILA cores in
compositions, which is shown in the next section.

The debug composition feature <a name="debug-feature"/>
-----------------------------

The `debug` feature can be added to any composition and can be configured in two
modes: `interfaces` mode and `nets` mode. The `interfaces` mode instantiates a
System ILA core during the generation of the Architecture. Its primary use is to
"listen" to transactions on AXI4 interfaces. To connect to an AXI4 interface,
you need to specify the (fully qualified) bus interface port and the
corresponding clock and reset pins. Example:

```
tapasco compose [precision_counter x 1] @ 100 Mhz --features 'Debug { interfaces: "{{/arch/target_ip_00_000/S00_AXI /arch/target_ip_00_000/s00_axi_aclk /arch/target_ip_00_000/s00_axi_aresetn }}" }'
```

This will attach a System ILA to the `S00_AXI` interface nets of the first
`precision_counter` instance.

The `nets` mode allows to connect to arbitrary nets (including nets within IP
cores which would not be visible in IP integrator). For this reason, it requires
two synthesis runs: First, the whole design without the ILA must be synthesized
into a netlist; then the ILA core is instantiated using a constraints file. This
requires a re-run of synthesis to take effect. Nets can be specified with
wildcards, patterns can be tested on a synthesized design using the Tcl console
in Vivado (use `get_bd_nets <pattern>` to test the pattern).

Example:

```
tapasco compose [precision_counter x 1] @ 100 Mhz --features 'Debug { nets: "{system_i/arch/target_ip_00_000/* system_i/arch/irq*}" }'
```

This would attach an ILA core to all nets attached to the first instance of the
first PE and all interrupt outputs of the entire Architecture. 

Exercise an ILA <a name="use-ila"/>
---------------

You can connect to the generated ILAs by starting Vivado by opening the
`microarch.xpr` in the subdirectory of the composition (probably below `bd`).
Open "Hardware Manager", connect to the board and the ILAs should already be
visible. Consult the Xilinx user guides for more information on ILA debugging.


Hunting software bugs <a name="sw-bugs"/>
---------------------

Software is slightly easier to debug. First step should be to build all modules
and libraries, as well as the application, in _debug mode_. This can be achieved
by either

```
tapasco-build-libs --rebuild --mode debug
```

This will rebuild the TaPaSCo libraries with logging facilities enabled. Logging
is controlled via **environment variables** listed in the table below.

|:**Env Var**           |:**Description**                                      |
|-----------------------|------------------------------------------------------|
| `LIBTAPASCO_DEBUG`    | Bitfield that enables/disables logging of subsystems |
|                       | in `libtapasco`, see below for details, -1 for all.  |
| `LIBPLATFORM_DEBUG`   | Bitfield that enables/disables logging of subsystems |
|                       | in `libplatform`, see below for details, -1 for all. |
| `LIBTAPASCO_LOGFILE`  | Redirects logging output from `libtapasco` to the    |
|                       | file specified here; can use absolute paths.         |
| `LIBPLATFORM_LOGFILE` | Redirects logging output from `libplatform` to the   |
|                       | file specified here; can use absolute paths.         |

The `LIBTAPASCO_DEBUG` bitfield enables logging in specific subsystems. Current
implementation is defined in the `LIBTAPASCO_LOGLEVELS` macro in
[tapasco_logging.h](arch/common/include/tapasco_logging.h). For reference, the
following bits are defined as of the time of writing:

|:**Bit #**|:**Description**                                                   |
|---------:|-------------------------------------------------------------------|
| 0        | _Reserved_	                                                       |
| 1        | Initialization - startup messages                                 |
| 2        | Device - interactions with the kernel module                      |
| 3        | Scheduler - TaPaSCo software scheduler                            |
| 4        | Interrupts                                                        |
| 5        | Memory                                                            |
| 6        | Function - hardware registers and enumeration                     |
| 7        | Status - TaPaSCo status core                                      |

The `LIBPLATFORM_DEBUG` bitfield enables logging from specific `libplatform`
subsystems. Current implementation is defined in the `LIBPLATFORM_LOGLEVELS`
macro in
[platform_logging.h](platform/common/include/platform_logging.h). For reference,
the following bits are defined as of the time of writing:

|:**Bit #**|:**Description**                                                   |
|---------:|-------------------------------------------------------------------|
| 0        | _Reserved_	                                                       |
| 1        | Initialization - startup messages                                 |
| 2        | Memory management                                                 |
| 3        | Memory allocator                                                  |
| 4        | Control Space interactions                                        |
| 5        | Interrupts                                                        |
| 6        | DMA related                                                       |

As a safe bet, simply use `-1` to activate all logging subsystems. Note that the
logging system is designed to be as unintrusive as possible; string operations
are always costly and _will_ affect your runtime, but TaPaSCo attempts to
minimize the impact by moving logging to a separate thread and keeping buffers
that will only flush occasionally. So you shouldn't expect the log output to
appear immediately - by default, TaPaSCo installs a signal handler that will
catch a `KILL` signal and flush logs, so you can use that to if your application
is stuck.

The tapasco-debug tool
----------------------

There's a versatile debugging tool that comes with TaPaSCo and is automatically
build by `tapasco-build-libs` called `tapasco-debug`. It can be used to inspect
bitstreams, peek and poke registers and perform basic functionality tests. One
of its most common features is the _kernel map_, which displays the function ids
of PEs in all virtual slots in the currently loaded bitstream, as well as host,
design and memory clock frequencies.

The tapasco-benchmark tool
--------------------------

Another tool that can sometimes be useful is `tapasco-benchmark` (also
automatically build by `tapasco-build-libs`). It performs a basic performance
evaluation of the Platform and stores the results in a JSON file. This file can
in turn be used by the design space exploration. Each Platform comes with a
pre-computed results file, e.g., [pynq](platform/pynq.benchmark). However, to
calibrate the DSE it may be useful to update the benchmarks on your own system.

**WARNING**: Make sure you've built everything in _release_ mode before updating
the benchmark data!

`tapasco-benchmark` requires a bitstream with at least one PE for a kernel with
id 14 - the counter. Counters are simple: Arg#0 is the number of clock cycles to
wait before raising the interrupt - that's it. One example of such a kernel is
[precision_counter](common/ip/precision_counter_1.0). You can build and import
it like this:

```
cd $TAPASCO_HOME/common/ip && \
zip -r precision_counter.zip precision_counter_v1.0 && \
tapasco import $(TAPASCO_HOME)/common/ip/precision_counter.zip as 14
```

Alternative implementations are also available, e.g., [counter](kernel/counter),
which is an Vivado HLS implementation of the same functionality. It can be built
using:

```
tapasco hls counter
```