Yes, I think the separation of generated code will need to occur prior to completing the merge and switching the web site to the main repo.
There should be no reason to check generated documentation into either of the repos/branches. Please see as an example how this was solved in Flink, using the ASF buildbot
Documentation per version/release, for example:
The buildbot configuration is here (requires committer access):
Last time I talked with Scott I brought this idea in. I believe the plan was either to publish compiled site to website directly, or keep it in separate storage from apache/beam repo.
One of the main reasons not to check in compiled version of website is that every developer will have to pull all the versions of website every time they clone repo, which is not that good of an idea to do.
Pablo, the docs are generated into versioned paths, e.g., https://beam.apache.org/documentation/sdks/javadoc/2.5.0/
so tags are not necessary?
Also, once apache/beam-site is merged with apache/beam the release branch should have the relevant docs (although perhaps it's better to put them in a different repo or storage system).
Thomas, I would very much like to not have javadoc/pydoc generation be part of the website review process, as it takes up a lot of time when changes are staged (10s of thousands of files), especially when a PR is updated and existing staged files need to be deleted.
+1 For removing old documentation.
@Thomas: Migration work is in backlog and will be picked up in near time.
Isn't it part of the beam-site changes that we will no longer check in generated documentation into the repository? Those can be generated and deployed independently (when a commit to a branch occurs), such as done in the Apex and Flink projects.
Is it worth adding a tag / branch to the repositories every time we make a release, so that people are able to dive in and find the docs?
I would guess that users are still using some of these old releases. It is unclear from Beam website which releases are still supported or not. It probably makes sense to drop documentation for releases < 2.0. (I would suggest keeping docs for 2.0). For the future I can work on updating the Beam website to clarify the state of each release.