Commit dcc6c418 authored by Jens Korinth's avatar Jens Korinth
Browse files

Squashed commit of the following:

commit 802a3eea
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:58:22 2018 +0100

    CI: Reactivate everything, with new compose-features stage

commit d0fb3e37
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:55:43 2018 +0100

    F'in yml

commit 6c8e648d
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:53:22 2018 +0100

    YML debugging

commit 155ed589
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:51:22 2018 +0100

    YML debugging

commit 4a6408b6
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:49:36 2018 +0100

    YML debugging

commit 1b09ffb8
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:47:22 2018 +0100

    Distinguish compose with and without features in jobs

commit ec2af8ed
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:43:24 2018 +0100

    Reactivate compose jobs

commit 09d289a8
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 12:41:17 2018 +0100

    Deactivate all jobs but compose without features

commit 40c7cba6
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 11:34:15 2018 +0100

    Fix potential hanging LogTrackingFileWatcher in EvaluateIP

commit f31d740c
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 11:19:20 2018 +0100

    Temporarily increase logging of file watcher (again)

commit f12a9074
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 10:45:05 2018 +0100

    Fix truncated log output in verbose mode

    * when an activity exits quickly with an error, verbose mode would often
      omit the most important last few lines of the log
    * reason: flushing the data to disk takes longer than for the Tapasco
      threads to die, thus LogTrackingFileWatcher exits before lines appear
    * workaround: when both waitingFor and files are empty, MultiFileWatcher
      now waits one more iteration before exiting, which seems to suffice

commit f9148c81
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 10:22:09 2018 +0100

    Squashed commit of the following:

    commit d3245516
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Thu Jan 25 10:19:19 2018 +0100

        Closes #149 - Zedboard Synthesis fails for 2017.3 and 2017.4

        * improved sys clock detection by checking available interfaces via
          get_board_part_interfaces, instead of trying sys_diff_clock first
        * also removed second warning when get_bd_pins returns nothing
        * removed old, unused platform code

    commit 9a0fb9b0
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Thu Jan 25 10:18:03 2018 +0100

        Use Arty-Z7-20 board file from Digilent for PyNQ

        * contains the manually set top.xdc directives
        * identical to Pynq, except for peripheral components

    commit f6a25afc
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Thu Jan 25 10:16:28 2018 +0100

        Update board definition for zedboard

        * Digilent has newer def of ZedBoard, using that automatically now
        * imported via MYVIVADO / XILINX_PATH env vars

    commit a29aa6cc
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Thu Jan 25 10:14:39 2018 +0100

        Fix minor bug in PS7 instantiation routine

    commit c782eadc
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Thu Jan 25 10:13:58 2018 +0100

        Remove 2016.4 specific code from design.master.tcl.template

commit f873c47b
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 25 08:23:54 2018 +0100

    Closes #137 for HLS

commit 88259aa0
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 17:01:44 2018 +0100

    Activate pipelines on branch gitlab-ci

commit d09278de
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 16:58:38 2018 +0100

    Deactivate sbt-prepare on all except master and 20xx.x branches

commit 9df2a6a6
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 16:54:40 2018 +0100

    Another fix regarding precision_counter

commit 30f9ac5b
Merge: c0ed8ed6 fc930ec8
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 16:53:33 2018 +0100

    Merge branch 'pe-local-memories' of git:tapasco/tapasco into gitlab-ci

commit c0ed8ed6
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 16:52:05 2018 +0100

    Remove precision_counter from composition

    * caches are not always available, so precision_counter is not
      available, causing runs to fail
    * annoying, removed precision_counter for now

commit 35b46901
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 15:56:50 2018 +0100

    Restrict CI pipelines to master and 20xx.x branches

commit 0a0dc675
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 15:45:13 2018 +0100

    Activate verbose output in CI HLS jobs

commit 240ad939
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 15:33:48 2018 +0100

    Test of generated yml

commit 4f684b2a
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 15:33:32 2018 +0100

    Remove support for Vivado 2016.4

commit be0a4b94
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 14:54:25 2018 +0100

    Pull tapasco-status 1.21

    * Chisel-generated Verilog is flattened into single module to avoid
      Verilog name conflicts

commit d668d82b
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 14:52:30 2018 +0100

    Fix bug in address map construction

    * internal master-slave connections do not appear in get_address_map
    * so their segments were not mapped, resulting in errors, e.g., for DMA
    * fix: when address map does not contain interfaces, it will try to
      deduce range and offset from properties of the segment instead of
      failing

commit 86b53707
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 24 13:45:58 2018 +0100

    Squashed commit of the following:

    commit 39e7a1cb
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Wed Jan 24 13:44:20 2018 +0100

        Bugfix in ZC706 fancontrol plugin

    commit 6a06399b
    Author: Lukas Sommer <lukas.sommer.mail@gmail.com>
    Date:   Wed Jan 24 12:39:49 2018 +0100

        Moved filter condition for active-high resets to correct command;

    commit cba13f81
    Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
    Date:   Wed Jan 24 11:28:40 2018 +0100

        Arch: Fix bug in PE reset connections

commit c830c899
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 09:00:20 2018 +0100

    Improve locking behavior of MultiFileWatcher

commit 58542825
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 08:22:55 2018 +0100

    Reactivate verbose mode in compose

commit b97700d3
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 06:31:23 2018 +0100

    Remove --maxTasks from HLS and Import

commit 25c503b6
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 06:25:15 2018 +0100

    Deactivate resource logging, increase tasks for HLS and import

commit ac61a13d
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 06:19:29 2018 +0100

    Remove tapasco-status building from sbt-prepare job

commit 4c6f2657
Merge: 3691ea58 d1f36ab4
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Tue Jan 23 06:15:50 2018 +0100

    Merge branch 'pe-local-memories' of esagitlab:tapasco/tapasco into gitlab-ci

commit 3691ea58
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Mon Jan 22 14:33:00 2018 +0100

    Change .gitlab-ci.yml to use new maxTasks param

commit ea7c3934
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Mon Jan 22 14:30:06 2018 +0100

    Closes #147 - Implement maxTasks option

    * now supports --maxTasks command line / JSON option to limit the number
      of parallel tasks executed by TaPaSCo

commit f8d090ae
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Sun Jan 21 15:32:51 2018 +0100

    Temporarily increase logging to debug OOM problems

commit 3cb861b2
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Sun Jan 21 12:46:13 2018 +0100

    Limit threads to 1 to remove oom errors

commit c02630a5
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 18:36:37 2018 +0100

    Adds artifact passing between hls and compose

commit e2c63695
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 18:02:27 2018 +0100

    Changes for shared caches

commit 82d85c69
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 17:11:36 2018 +0100

    Try to cache as much as possible

commit 36be0ac0
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 17:04:08 2018 +0100

    Removes /cache

commit 6a046f78
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 17:03:27 2018 +0100

    Adds cache_global

commit fce5833e
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 17:00:42 2018 +0100

    Adds /opt/cad ls for debugging

commit 0bb7f526
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 16:52:50 2018 +0100

    Adds cache test

commit 5e410f65
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 16:24:17 2018 +0100

    Removes -v so tapasco finishes

commit 532b1ba1
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 16:10:45 2018 +0100

    Reverts changes to default image

commit 25d70ece
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 16:09:42 2018 +0100

    Replaces _JAVA_OPTIONS with SBT_OPTS

        - _JAVA_OPTIONS seems to annoy Vivado HLS and results in random
        crashes

commit b1bbfe5f
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 14:48:33 2018 +0100

    Compile only one platform for hls

commit 90adf995
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 14:27:31 2018 +0100

    He said I should add an s

commit 16664088
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 14:26:19 2018 +0100

    Adds artifact for hls builds

commit 53f0ae16
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 13:18:53 2018 +0100

    Mkaes hls verbose for debugging

commit 2becdf3c
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 13:03:15 2018 +0100

    Removes space (and time)

commit 3a444c26
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 12:03:51 2018 +0100

    Adds check for cache success

commit cd75f301
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 12:01:16 2018 +0100

    Checks if vivado is included properly

commit 425b8e98
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 11:48:36 2018 +0100

    Changes sbt-prepare cache policy to push

commit c2f3b5ac
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 11:47:09 2018 +0100

    Replaces artifacts with cache

        - For now only sbt-prepare is shared for further jobs
        - There should be further sharing between Import and Compose

commit b2b2d9b6
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Fri Jan 19 10:52:51 2018 +0100

    Revert "Fixes #145 - Building status core fails"

    This reverts commit 32047b00.
    The error this commit fixes was already fixed in a new Dockerfile
    version.

commit 32047b00
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Fri Jan 19 10:40:58 2018 +0100

    Fixes #145 - Building status core fails

commit 55718f4f
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 17:42:33 2018 +0100

    Removes import as dependency for compose

commit 89b6f957
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 17:23:25 2018 +0100

    Reduces artifact size by compressing them

commit 17a28552
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 16:06:44 2018 +0100

    Source Vivado

        For whatever reason .bashrc is not evaluated...

commit 2cd05d74
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 15:49:46 2018 +0100

    Revert back to targetted artifacts because of size constraints

commit fb4284aa
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 15:36:41 2018 +0100

    Get all untracked files for prepare stage

commit 2f87a5f7
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 15:25:29 2018 +0100

    Bin is needed as artifact as well

commit 29b4cd38
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 15:07:22 2018 +0100

    Limit artifacts to .ivy2 and .sbt

commit 5524d39a
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 14:55:06 2018 +0100

    Adapts import-template to new format

commit c53313a0
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Thu Jan 18 14:52:57 2018 +0100

    Used tapasco image and artifact sbt

commit 7fd62db1
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:58:10 2018 +0100

    Implement import task as pipeline stage

commit 6e9c33a9
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:51:34 2018 +0100

    Improve logging in Zynq platform Tcl

commit e1758dbb
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:51:02 2018 +0100

    Improve logging in platform common Tcl

commit 090f4481
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:50:32 2018 +0100

    Improve logging in tapasco::ip

commit a6e9559d
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:49:58 2018 +0100

    Activate PE-local memory capability by default

commit e3682565
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 13:49:40 2018 +0100

    Bugfix in Arch address map for unconnected masters

commit b8b5dd0a
Merge: dd6ac83b 8d677bb7
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 07:59:19 2018 +0100

    Pull tapasco-status

    Merge commit '8d677bb7' into gitlab-ci

commit 8d677bb7
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Thu Jan 18 07:58:18 2018 +0100

    Squashed 'common/ip/tapasco_status/' changes from e209f949..3fd53e7

    3fd53e7 Pull chisel-axi
    41e37e7 Squashed 'axi/' changes from b8f4c554..01fad68
    88624e0 Pull chisel-packaging
    872f551 Squashed 'packaging/' changes from 134b2f62..c22243b
    f94b6de3 Remove caching of ivy repo from pipeline
    43c331dc Pull chisel-axiutils
    5937e2aa Implement cap0 bitfield
    26d61dd6 Bugfix in pipeline
    1030ffe5 Cache ivy2 repo in pipeline builds
    14876b2e Implement support for capability field in Status Core
    2a3e6856 Fix removed '<<=' sbt operator
    bccc8a73 Run sbt test in GitLab pipeline
    17e1a3a7 Fix bug concerning empty slots
    5a089419 Ignore compiled python scripts in .gitignore
    0f0a2d84 Update packaging to GitHub-version of Chisel3
    a162cfae Update miscutils to GitHub-version of Chisel3
    f0265156 Remove ununsed Scalactic dep
    d146b992 Rename RegisterFile saxi port to s_axi

    git-subtree-dir: common/ip/tapasco_status
    git-subtree-split: 3fd53e7038ab7e1ff485eee94c3516f72b9604ea

commit dd6ac83b
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 21:31:34 2018 +0100

    Adds verbose flag for compose debugging

commit d5e9bc89
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 21:06:26 2018 +0100

    Fixes job naming in yaml

commit 9b9304ee
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 21:04:59 2018 +0100

    Use artifacts to avoid reevaluation

commit fd6843ad
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 20:45:10 2018 +0100

    Removes verbose flag to avoid lock ups

commit 23f78ad7
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 20:12:12 2018 +0100

    Fixes gitlab-ci with Centos7

commit 4c73c789
Author: Jaco Hofmann <hofmann@esa.tu-darmstadt.de>
Date:   Wed Jan 17 19:03:03 2018 +0100

    Change image type to centos

commit 0b823caa
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 18:22:30 2018 +0100

    Implement stages and HLS

commit 8e810c8c
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 16:33:51 2018 +0100

    Fix

commit e34de5b8
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 16:32:18 2018 +0100

    Fix

commit 0dbd2a57
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 16:23:07 2018 +0100

    Fix

commit 408774c9
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 16:19:22 2018 +0100

    Fix setup script

commit 38820724
Author: Jens Korinth <jk@esa.cs.tu-darmstadt.de>
Date:   Wed Jan 17 16:16:08 2018 +0100

    Implement automated regression tests
parent d3245516
Pipeline #201 passed with stage
in 3 minutes and 18 seconds
This diff is collapsed.
# create a dictionary of compatible VLNVs
source $::env(TAPASCO_HOME)/common/common_ip.tcl
dict set stdcomps dualdma vlnv "esa.informatik.tu-darmstadt.de:user:dual_dma:1.7"
dict set stdcomps clk_wiz vlnv "xilinx.com:ip:clk_wiz:5.3"
image: tapasco
variables:
XILINXD_LICENSE_FILE: "/opt/cad/keys/xilinx"
before_script:
- uname -a
- echo $SHELL
- echo $PWD
- source $PWD/setup.sh
- source /root/.sdkman/bin/sdkman-init.sh
- export SBT_OPTS="-Dsbt.global.base=$TAPASCO_HOME/.sbt/ -Dsbt.ivy.home=$TAPASCO_HOME/.ivy2/ -Divy.home=$TAPASCO_HOME/.ivy2/"
stages:
- toolchain-test
- hls
- import
- compose
- compose-features
- dse
scala-test:
stage: toolchain-test
script:
- sbt clean test
.template: &template
only:
- "gitlab-ci"
- "master"
- /20\d\d\.\d/
sbt-prepare:
<<: *template
stage: toolchain-test
cache:
untracked: true
key: "$CI_COMMIT_SHA-$CI_COMMIT_REF_NAME"
policy: push
script:
- sbt assembly
.hls-template: &hls-definition
<<: *template
stage: hls
dependencies:
- sbt-prepare
cache:
untracked: true
key: "$CI_COMMIT_SHA-$CI_COMMIT_REF_NAME"
policy: pull
artifacts:
when: always
expire_in: 1 hours
paths:
- core
script:
- source /opt/cad/$VIVADO_OFFSET/settings64.sh
- vivado -version
- |
if [ ! -d ".ivy2" ]; then
echo "Cache not available"
sbt assembly
fi
- tapasco -v hls arraysum, arrayinit -a $ARCHS -p $PLATFORMS
.import-template: &import-definition
<<: *template
stage: import
dependencies:
- sbt-prepare
cache:
untracked: true
key: "$CI_COMMIT_SHA-$CI_COMMIT_REF_NAME"
policy: pull
script:
- source /opt/cad/$VIVADO_OFFSET/settings64.sh
- |
if [ ! -d ".ivy2" ]; then
echo "Cache not available"
sbt assembly
fi
- cd $TAPASCO_HOME/common/ip && zip -r precision_counter.zip precision_counter*
- tapasco -v import $TAPASCO_HOME/common/ip/precision_counter.zip as 14 -a $ARCHS -p $PLATFORMS
.compose-template: &compose-definition
<<: *template
stage: compose
cache:
untracked: true
key: "$CI_COMMIT_SHA-$CI_COMMIT_REF_NAME"
policy: pull
script:
- source /opt/cad/$VIVADO_OFFSET/settings64.sh
- |
if [ ! -d ".ivy2" ]; then
echo "Cache not available"
sbt assembly
fi
- tapasco -v --maxTasks 1 compose [arraysum x 1, arrayupdate x 1] @ 100 MHz -a $ARCHS -p $PLATFORMS
.compose-features-template: &compose-features-definition
<<: *template
stage: compose-features
cache:
untracked: true
key: "$CI_COMMIT_SHA-$CI_COMMIT_REF_NAME"
policy: pull
script:
- source /opt/cad/$VIVADO_OFFSET/settings64.sh
- |
if [ ! -d ".ivy2" ]; then
echo "Cache not available"
sbt assembly
fi
- tapasco -v --maxTasks 1 compose [arraysum x 1, arrayupdate x 1] @ 100 MHz -a $ARCHS -p $PLATFORMS --features "$FEATURES"
.TH tapasco 1 2017.1 MAN(1)
.TH tapasco 1 2018.1 MAN(1)
.SH SYNOPSIS
tapasco [global option]* [job]*
or: tapasco \-h | \-\-help [TOPIC]
......@@ -80,7 +80,13 @@ Execute all jobs in parallel (careful!)
.RS
\-\-maxThreads NUM
.RS
Limit internal parallelism of activities (e.g., Vivado) to the given number of threads.
Limit internal parallelism of tasks (e.g., Vivado) to the given number of threads.
.RE
.RE
.RS
\-\-maxTasks NUM
.RS
Limit max. number tasks executed in parallel to the given number
.RE
.RE
.SH BULK IMPORT JOB
......@@ -612,6 +618,12 @@ any quoted or unquoted string containing additional information about the core
.RE
.RE
.RS
\-\-skipEvaluation
.RS
do not perform out\-of\-context synthesis to get resource estimates, only import
.RE
.RE
.RS
\-\-averageClockCycles N
.RS
any integer > 0; number of clock cycles in an "average" execution of the PE; used to estimate the maximal throughput
......
......@@ -237,9 +237,16 @@ namespace eval platform {
set sintf [get_bd_intf_pins -quiet -of_objects $seg]
if {[catch {dict get $map $intf}]} {
if {[catch {dict get $map $sintf}]} {
error "neither $intf nor $sintf were found in address map for $seg: $::errorInfo"
puts " neither $intf nor $sintf were found in address map for $seg: $::errorInfo"
puts " assuming internal connection, setting values as found in segment:"
set range [get_property RANGE $seg]
puts " range: $range"
set offset [get_property OFFSET $seg]
puts " offset: $offset"
set me [dict create "range" $range "offset" $offset "space" $space seg "$seg"]
} else {
set me [dict get $map $sintf]
}
set me [dict get $map $sintf]
} else {
set me [dict get $map $intf]
}
......
......@@ -40,6 +40,7 @@ namespace eval platform {
}
proc get_address_map {{pe_base ""}} {
set max32 [expr "1 << 32"]
set max64 [expr "1 << 64"]
if {$pe_base == ""} { set pe_base [get_pe_base_address] }
set peam [::arch::get_address_map $pe_base]
......@@ -114,7 +115,7 @@ namespace eval platform {
puts "Unused Interrupts: 1, 2, 3 are tied to 0"
connect_bd_net [get_bd_pin -of_object $irq_unused -filter {NAME == "dout"}] [get_bd_pin -of_objects $irq_concat_ss -filter {NAME == "In1"}]
for {set i 0} {$i < 4} {incr i} {
set port [create_bd_pin -dir I -type intr "irq_$i"]
set port [create_bd_pin -from 127 -to 0 -dir I -type intr "intr_$i"]
connect_bd_net $port [get_bd_pin $irq_concat_ss/[format "In%d" [expr "$i + 2"]]]
}
......
......@@ -13,16 +13,16 @@
<appender-ref ref="STDOUT" />
</root>
<!--<appender name="STDOUT-DETAIL" class="ch.qos.logback.core.ConsoleAppender">
<appender name="STDOUT-DETAIL" class="ch.qos.logback.core.ConsoleAppender">
<withJansi>true</withJansi>
<encoder>
<pattern>[%blue(%d{HH:mm:ss} &lt;%thread: %c{0}&gt;) %highlight(%level)] %msg%n</pattern>
</encoder>
</appender>
<logger name="de.tu_darmstadt.cs.esa.tapasco.activity.composers" level="trace">
<logger name="de.tu_darmstadt.cs.esa.tapasco.filemgmt.LogTrackingFileWatcher" level="trace">
<appender-ref ref="STDOUT-DETAIL" />
</logger>-->
</logger>
<!--<logger name="de.tu_darmstadt.cs.esa.tapasco.itapasco.view.detail.LogTrackingPanel" level="trace">
<appender-ref ref="STDOUT-DETAIL" />
......
......@@ -68,52 +68,56 @@ object Tapasco {
// scalastyle:off cyclomatic.complexity
// scalastyle:off method.length
def main(args: Array[String]) {
implicit val tasks = new Tasks
val ok = try {
val cfgargs = if (args.length > 0 && args(0).toLowerCase.equals("itapasco")) args.drop(1) else args
// try to parse all arguments
val c = CommandLineParser(cfgargs mkString " ")
logger.debug("parsed config: {}", c)
if (c.isRight) {
// get parsed Configuration
implicit val cfg = c.right.get
// dump config and exit, if dryRun is selected
cfg.dryRun foreach (dryRun _)
// else continue ...
logger.trace("configuring FileAssetManager...")
FileAssetManager(cfg)
logger.trace("SLURM: {}", cfg.slurm)
if (cfg.slurm) Slurm.enabled = cfg.slurm
FileAssetManager.start()
logger.trace("parallel: {}", cfg.parallel)
cfg.logFile map { logfile: Path => setupLogFileAppender(logfile.toString) }
logger.info("Running with configuration: {}", cfg.toString)
implicit val exe = ExecutionContext.fromExecutor(new java.util.concurrent.ForkJoinPool(UNLIMITED_THREADS))
def get(f: Future[Boolean]): Boolean = { Await.ready(f, duration.Duration.Inf); f.value map (_ getOrElse false) getOrElse false }
val cfgargs = if (args.length > 0 && args(0).toLowerCase.equals("itapasco")) args.drop(1) else args
// try to parse all arguments
val c = CommandLineParser(cfgargs mkString " ")
logger.debug("parsed config: {}", c)
val ok = if (c.isRight) {
implicit val tasks = new Tasks(c.right.get.maxTasks)
// get parsed Configuration
implicit val cfg = c.right.get
// dump config and exit, if dryRun is selected
cfg.dryRun foreach (dryRun _)
// else continue ...
logger.trace("configuring FileAssetManager...")
FileAssetManager(cfg)
logger.trace("SLURM: {}", cfg.slurm)
if (cfg.slurm) Slurm.enabled = cfg.slurm
FileAssetManager.start()
logger.trace("parallel: {}", cfg.parallel)
cfg.logFile map { logfile: Path => setupLogFileAppender(logfile.toString) }
logger.info("Running with configuration: {}", cfg.toString)
def get(f: Future[Boolean]): Boolean = { Await.ready(f, duration.Duration.Inf); f.value map (_ getOrElse false) getOrElse false }
try {
if (cfg.parallel) {
implicit val exe = ExecutionContext.fromExecutor(new java.util.concurrent.ForkJoinPool(cfg.maxTasks getOrElse UNLIMITED_THREADS))
runGui(args) || (cfg.jobs map { j => Future { jobs.executors.execute(j) } } map (get _) fold true) (_ && _)
} else {
runGui(args) || (cfg.jobs map { jobs.executors.execute(_) } fold true) (_ && _)
}
} else {
logger.error("invalid arguments: {}", c.left.get.toString)
logger.error("run `tapasco -h` or `tapasco --help` to get more info")
} catch { case ex: Exception =>
logger.error(ex.toString)
logger.error("Stack trace: {}", ex.getStackTrace() map (_.toString) mkString "\n")
false
} finally {
FileAssetManager.stop()
tasks.stop()
}
} catch { case ex: Exception =>
logger.error(ex.toString)
logger.error("Stack trace: {}", ex.getStackTrace() map (_.toString) mkString "\n")
} else {
logger.error("invalid arguments: {}", c.left.get.toString)
logger.error("run `tapasco -h` or `tapasco --help` to get more info")
false
} finally {
FileAssetManager.stop()
tasks.stop()
}
logger.debug("active threads: {}", Thread.activeCount())
if (Thread.activeCount() > 0) {
import scala.collection.JavaConverters._
val m = Thread.getAllStackTraces().asScala
m.values foreach { strace => logger.debug(strace mkString scala.util.Properties.lineSeparator) }
}
if (! ok) {
logger.error("TaPaSCo finished with errors")
sys.exit(1)
......
......@@ -137,12 +137,12 @@ object EvaluateIP {
logger.trace("Vivado command: {}", vivadoCmd mkString " ")
cfg.verbose foreach { _ => lt.closeAll }
// execute Vivado (max runtime: 1d)
val r = InterruptibleProcess(Process(vivadoCmd, files.baseDir.toFile),
waitMillis = Some(24 * 60 * 60 * 1000)).!(io)
cfg.verbose foreach { _ => lt.closeAll }
if (r == InterruptibleProcess.TIMEOUT_RETCODE) {
logger.error("%s: Vivado timeout error".format(runPrefix))
} else {
......
......@@ -68,6 +68,7 @@ private object VivadoHighLevelSynthesis extends HighLevelSynthesizer {
), script.getParent.toFile), waitMillis = Some(24 * 60 * 60 * 1000))
.!(ProcessLogger(line => logger.trace("Vivado HLS: {}", line),
line => logger.trace("Vivado HLS ERR: {}", line)))
lt.closeAll
logger.debug("Vivado HLS finished with exit code %d".format(vivadoRet))
vivadoRet match {
case 0 =>
......
......@@ -50,6 +50,8 @@ trait Configuration {
def parallel(enabled: Boolean): Configuration
def maxThreads: Option[Int]
def maxThreads(mt: Option[Int]): Configuration
def maxTasks: Option[Int]
def maxTasks(mt: Option[Int]): Configuration
def dryRun(cfg: Option[Path]): Configuration
def dryRun: Option[Path]
def verbose(mode: Option[String]): Configuration
......
......@@ -44,6 +44,7 @@ private case class ConfigurationImpl (
slurm: Boolean = false,
parallel: Boolean = false,
maxThreads: Option[Int] = None,
maxTasks: Option[Int] = None,
dryRun: Option[Path] = None,
verbose: Option[String] = None,
jobs: Seq[Job] = Seq()
......@@ -64,6 +65,7 @@ private case class ConfigurationImpl (
def slurm(enabled: Boolean): Configuration = this.copy(slurm = enabled)
def parallel(enabled: Boolean): Configuration = this.copy(parallel = enabled)
def maxThreads(mt: Option[Int]): Configuration = this.copy(maxThreads = mt)
def maxTasks(mt: Option[Int]): Configuration = this.copy(maxTasks = mt)
def dryRun(cfg: Option[Path]): Configuration = this.copy(dryRun = cfg)
def verbose(mode: Option[String]): Configuration = this.copy(verbose = mode)
def jobs(js: Seq[Job]): Configuration = this.copy(jobs = js)
......
......@@ -83,6 +83,7 @@ private object PrettyPrinter {
"Slurm = " + c.slurm,
"Parallel = " + c.parallel,
"MaxThreads = " + (c.maxThreads getOrElse "unlimited"),
"MaxTasks = " + (c.maxTasks getOrElse "unlimited"),
"Jobs = " + c.jobs
) mkString NL
......
......@@ -293,6 +293,7 @@ package object json {
(JsPath \ "Slurm").readNullable[Boolean].map (_ getOrElse false) ~
(JsPath \ "Parallel").readNullable[Boolean].map (_ getOrElse false) ~
(JsPath \ "MaxThreads").readNullable[Int] ~
(JsPath \ "MaxTasks").readNullable[Int] ~
(JsPath \ "DryRun").readNullable[Path] ~
(JsPath \ "Verbose").readNullable[String] ~
(JsPath \ "Jobs").read[Seq[Job]]
......@@ -308,6 +309,7 @@ package object json {
(JsPath \ "Slurm").write[Boolean] ~
(JsPath \ "Parallel").write[Boolean] ~
(JsPath \ "MaxThreads").writeNullable[Int] ~
(JsPath \ "MaxTasks").writeNullable[Int] ~
(JsPath \ "DryRun").writeNullable[Path].transform((js: JsObject) => js - "DryRun") ~
(JsPath \ "Verbose").writeNullable[String] ~
(JsPath \ "Jobs").write[Seq[Job]]
......
......@@ -39,14 +39,14 @@ class MultiFileWatcher(pollInterval: Int = MultiFileWatcher.POLL_INTERVAL) exten
* Add a file to the monitoring.
* @param p Path to file to be monitored.
*/
def +=(p: Path) { open(p) }
def +=(p: Path) { _waitingFor.synchronized { _waitingFor += p }; open(p) }
@inline def addPath(p: Path) { this += p }
/**
* Add a collection of files to the monitoring.
* @param ps Collection of Paths to files to be monitored.
*/
def ++=(ps: Traversable[Path]) { ps foreach (open _) }
def ++=(ps: Traversable[Path]) { ps foreach (this += _) }
@inline def addPaths(ps: Traversable[Path]) { this ++= ps }
/**
......@@ -74,21 +74,19 @@ class MultiFileWatcher(pollInterval: Int = MultiFileWatcher.POLL_INTERVAL) exten
private def open(p: Path): Boolean = {
val res = try {
_files += p -> new BufferedReader(new FileReader(p.toString))
_files.synchronized { _files += p -> new BufferedReader(new FileReader(p.toString)) }
logger.trace("opened {} successfully", p.toString)
true
} catch { case ex: java.io.IOException =>
logger.trace("could not open {}, will retry ({})", p: Any, ex: Any)
_waitingFor.synchronized { _waitingFor += p }
false
}
startWatchThread
res
}
private def close(p: Path): Unit = if (_files.contains(p)) {
_files -= p
} else if (_waitingFor.contains(p)) {
private def close(p: Path): Unit = {
_files.synchronized { _files -= p }
_waitingFor.synchronized { _waitingFor -= p }
}
......@@ -98,32 +96,36 @@ class MultiFileWatcher(pollInterval: Int = MultiFileWatcher.POLL_INTERVAL) exten
}
private def startWatchThread: Unit = {
if (_watchThread.get.isEmpty) {
logger.trace("starting file watch thread ...")
_watchThread.set(Some(new Thread(new Runnable {
def run() {
try {
while (! _files.isEmpty || ! _waitingFor.isEmpty) {
Thread.sleep(pollInterval)
val waits = _waitingFor.synchronized { _waitingFor.toList }
waits foreach { p =>
logger.trace("waiting for {}", p)
if (open(p)) _waitingFor.synchronized { _waitingFor -= p }
}
val files = _files.toMap
files foreach { case (p, br) =>
val lines = readFrom(br)
if (lines.length > 0) {
logger.trace("read {} lines from {}", lines.length, p)
publish(LinesAdded(p, lines))
}
logger.trace("starting file watch thread ...")
if (_watchThread.compareAndSet(None, Some(new Thread(new Runnable {
def run() {
try {
var lastWasEmpty = false
while (! _files.isEmpty || ! _waitingFor.isEmpty || ! lastWasEmpty) {
val waits = _waitingFor.synchronized { _waitingFor.toList }
val files = _files.synchronized { _files.toMap }
Thread.sleep(pollInterval)
waits foreach { p =>
logger.trace("waiting for {}", p)
if (open(p)) _waitingFor.synchronized { _waitingFor -= p }
}
val all_files = files ++ _files.synchronized { _files.toMap }
logger.trace("reading from files: {}", files)
files foreach { case (p, br) =>
val lines = readFrom(br)
if (lines.length > 0) {
logger.trace("read {} lines from {}", lines.length, p)
publish(LinesAdded(p, lines))
}
}
_watchThread.set(None)
} catch { case e: InterruptedException => _watchThread.set(None) }
}
})))
lastWasEmpty = files.isEmpty
}
_watchThread.set(None)
} catch { case e: InterruptedException => _watchThread.set(None) }
}
})))) {
_watchThread.get map (_.start)
Thread.sleep(100)
}
}
......@@ -139,6 +141,6 @@ object MultiFileWatcher {
/** Lines ls have been added to file at src. **/
final case class LinesAdded(src: Path, ls: Traversable[String]) extends Event
}
/** Default polling interval for files: once per second. **/
final val POLL_INTERVAL = 1000 // 1sec
/** Default polling interval for files: once every 5 seconds. **/
final val POLL_INTERVAL = 5000 // 5sec
}
......@@ -43,7 +43,8 @@ private object GlobalOptions {
longOption("logFile") |
longOption("parallel") |
longOption("slurm") |
longOption("maxThreads")
longOption("maxThreads") |
longOption("maxTasks")
).opaque("a global option")
def help: Parser[(String, String)] =
......@@ -93,8 +94,11 @@ private object GlobalOptions {
def maxThreads: Parser[(String, Int)] =
longOption("maxThreads", "MaxThreads") ~/ ws ~ posint ~ ws
def maxTasks: Parser[(String, Int)] =
longOption("maxTasks", "MaxTasks") ~/ ws ~ posint ~ ws
def globalOptionsSeq: Parser[Seq[(String, _)]] =
ws ~ (help | verbose | dirs | inputFiles | slurm | parallel | dryRun | maxThreads).rep
ws ~ (help | verbose | dirs | inputFiles | slurm | parallel | dryRun | maxThreads | maxTasks).rep
def globalOptions: Parser[Configuration] =
globalOptionsSeq map (as => mkConfig(as))
......@@ -115,6 +119,7 @@ private object GlobalOptions {
case ("ConfigFile", p: Path) => mkConfig(as, Some(loadConfigFromFile(p)))
case ("DryRun", p: Path) => mkConfig(as, Some(c getOrElse Configuration() dryRun Some(p)))
case ("MaxThreads", i: Int) => mkConfig(as, Some(c getOrElse Configuration() maxThreads Some(i)))
case ("MaxTasks", i: Int) => mkConfig(as, Some(c getOrElse Configuration() maxTasks Some(i)))
case ("Verbose", m: String) => mkConfig(as, Some(c getOrElse Configuration() verbose Some(m)))
case _ => c getOrElse Configuration()
}
......
......@@ -82,8 +82,9 @@ configuration via `tapasco -n config.json`.
Arg("--jobsFile FILE", "Path to Json file with Jobs array") &
Arg("--slurm", "Activate SLURM cluster execution (requires sbatch)") &
Arg("--parallel", "Execute all jobs in parallel (careful!)") &
Arg("--maxThreads NUM", "Limit internal parallelism of activities (e.g., Vivado)" ~
"to the given number of threads."))
Arg("--maxThreads NUM", "Limit internal parallelism of tasks (e.g., Vivado)" ~
"to the given number of threads.") &
Arg("--maxTasks NUM", "Limit max. number tasks executed in parallel to the given number"))
private def composition() = Section("Composition Syntax",
Block("A Composition specifies the number and kind of processing elements (PEs) that" ~
......@@ -341,7 +342,7 @@ configuration via `tapasco -n config.json`.
private def shortTopics() = Section("Help Topics", Block(helpTopics.keys.toSeq.sorted.mkString(", ")))
private def overview() =
Header("tapasco", 1, "2017-07-14", "2017.1") &
Header("tapasco", 1, "2018-01-22", "2018.1") &
Synopsis(" tapasco [global option]* [job]*" &
"or: tapasco -h | --help [TOPIC]" &
"")
......
......@@ -100,12 +100,13 @@ private class GenericTask(