Hi,
is this a typo, or does the model actually require 80GB of GPU RAM to do inference?
https://github.com/SAP-samples/sap-rpt-1-oss/blame/4310453f3bbb9c9c403b029e966565e7fe652d97/README.md#L56
And are there recommended specs for the faster, light-weight configuration (context size 2048, bagging factor 1)?