trade-tariff-backend: Reporting
Spreadsheet based reports of different datasets for both UK (CDS) and XI (Taric) tariffs are generated on a nightly basis.
These are generated within this application using workers scheduled to run on Sidekiq in the morning after the daily synchronisation of tariff data has occurred.
After generate, these files are uploaded to our Reporting S3 bucket.
These reports are generated daily on both UK and XI backend services, with the exception of Differences report, which is UK only and only generated on Mondays.
Manual regeneration
Regeneration of these files can be manually triggered from the UK and XI backends using a dedicated rake task - which will trigger running the worker job.
# bundle exec rake reporting:generate_daily_reports
Differences report
There is a separate report, referred to as the differences report or sometimes the 'potential issues report'
It is generated by a separate worker which triggers only on Mondays on the UK service, and is triggered 30 minutes after the completion of the UK daily reports. This delay is to both XI and UK have completed daily report generation.
It works by downloading CSVs of relevant data for both UK and XI from the Reporting bucket and generating an XLSX highlighting differences between the two.
This report is emailed automatically to HMRC.
It can be manually triggered on any day using a rake task
# bundle exec rake reporting:generate_differences_report
To generate without emailing the report to HMRC use the following
# bundle exec rake reporting:generate_differences_report NOEMAIL=true
Code
The code used to generate the various reports is held under app/lib/reporting with different reports following a common pattern of Reporting::<ReportName>.generate
The actual reports generated by each of the workers can be seen in the relevant workers, see