Great article! In the last section, you mentioned that notebooks are a good way of prototyping workflows, but I believe that notebooks are feasible in production pipelines. I’ve done so in a few projects where I split my pipeline in several small units where some of them are actual notebooks that generate HTML reports. I wrote a library to support such workflow (uses papermill under the hood): https://github.com/ploomber/ploomber

Example: https://ploomber.readthedocs.io/en/stable/auto_examples/reporting.html#sphx-glr-auto-examples-reporting-py