osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Migrating git repos to OpenDev


On Thu, Apr 11, 2019, at 3:21 PM, Clark Boylan wrote:
> On Thu, Mar 7, 2019, at 1:46 PM, James E. Blair wrote:
> > Hi,
> > 
> > As discussed in November[1], the OpenStack project infrastructure is
> > being rebranded as "OpenDev" to better support a wider community of
> > projects.
> > 
> > We are nearly ready to perform the part of this transition with the
> > largest impact: moving the authoritative git repositories for existing
> > projects.
> > 
> > In this email, I'd like to introduce the new hosting system we are
> > preparing, discuss the transition, and invite projects to work with us
> > on the logistics of the change.
> > 
> > Gerrit
> > ======
> > 
> > Gerrit is the core of our system and it will remain so in OpenDev.  As
> > part of this move, we will rename the gerrit server from
> > review.openstack.org to review.opendev.org.  As part of the transition,
> > we will automatically merge appropriate changes to all branches of all
> > repositories updating .gitreview and Zuul configuration files.  Any
> > further changes (README files, etc.) we expect to be made by individual
> > project contributors.
> > 
> > Repository Browsing
> > ===================
> > 
> > Currently our canonical *public* repository system is the cgit server at
> > https://git.openstack.org/ (and git.airshipit.org, git.starlingx.io, and
> > git.zuul-ci.org).  This is a load balanced cluster of several servers
> > which is designed to handle all the public git repository traffic, as it
> > scales much better than Gerrit (and has a more friendly domain name).
> > From a technical standpoint, it's excellent, but its usability could be
> > improved.
> > 
> > Therefore, as part of this transition, we will replace the cgit servers
> > with a new system based on Gitea.  Gitea is a complete development
> > collaboration system, but it's very flexible and will allow us to
> > disable components which we aren't using.  We will operate it in a
> > read-only configuration where it will act as the public mirror for
> > Gerrit.  The advantages it has over the current system are:
> > 
> > * Shorter domain name in project URLs:
> >   https://git.openstack.org/openstack/nova   vs
> >   https://opendev.org/openstack/nova
> > * Clone and browsing URLs are the same (with cgit, the browsing URL has
> >   an extra path component)
> > * More visually pleasing code browsing
> > * Integrated code searching
> > * Ability to highlight multiple lines in links
> > 
> > When we perform the transition we will install redirects from
> > git.openstack.org (and the other git sites) to opendev.org, and will
> > maintain those redirects for the foreseeable future.  We will construct
> > them so that even existing deep links to individual files in individual
> > commits to cgit will redirect to the correct location on opendev.org.
> > 
> > This system is up and running now with a live mirror of data from
> > Gerrit, and you can start testing it out today at https://opendev.org/
> > 
> > Please let us know if you encounter any problems.
> > 
> > If you would like to read more about the design of this system and the
> > transition, see the infra-spec[2].
> > 
> > GitHub
> > ======
> > 
> > Currently all OpenStack projects are replicated to GitHub.  We do not
> > plan on changing that during the transition, however, any projects
> > outside of the openstack*/ namespaces will not automatically be
> > replicated to GitHub, and we do not plan on adding that in the future.
> > We do, however, support projects using Zuul to run post-merge jobs to
> > push updates to GitHub or any other third-party mirrors with their own
> > credentials.  We will be happy to work with anyone interested in that to
> > help set up jobs to do so.
> > 
> > We are adopting this approach so that individual projects can have more
> > control over how they are represented in social media, and to give us
> > more flexibility in supporting our own organizational namespaces on
> > OpenDev without assuming they map directly to GitHub.
> > 
> > Eventually we plan on moving the OpenStack project to that system as
> > well and retiring direct replication from Gerrit to GitHub completely.
> > But we will defer that work until after this transition.
> > 
> > Logistics
> > =========
> > 
> > We can prepare much of the system in advance (as we have for the hosting
> > system on opendev.org), but the actual transition and renaming of the
> > Gerrit server will need to happen at once during an outage window.  We
> > need to schedule that outage and begin preparing for it.
> > 
> > Since all of the project git URLs are going to change (to replace
> > git.openstack.org with opendev.org and review.openstack.org with
> > review.opendev.org), we can additionally take the opportunity to
> > reorganize projects into different organizations.
> > 
> > For example, during the transition we will rename Zuul, and it's
> > associated projects, from the "openstack-infra" org to "zuul".  So their
> > new names will be "zuul/zuul", "zuul/nodepool", etc.
> > 
> > This is an excellent time for the rest of the OpenStack Foundation
> > pilot projects to do the same.
> > 
> > If the OpenStack project desires this, it would also be a good time to
> > move unofficial projects out of the openstack/ namespace.
> > 
> > Therefore, we need your help:
> > 
> > Action Items
> > ============
> > 
> > We need each of the following projects:
> > 
> >   * OpenStack
> >   * Airship
> >   * StarlingX
> >   * Zuul
> > 
> > To nominate a single point of contact to work with us on the transition.
> > It would be helpful for that person to attend the next (and possibly
> > next several) openstack infra team meetings in IRC [3].  We will work
> > with those people on scheduling the transition, as well as finalizing
> > the list of projects which should be renamed as part of the transition.
> > 
> > If you manage an unofficial project and would like to take the
> > opportunity to move or rename your project, please add it to this
> > ethercalc[4].
> > 
> > [1] 
> > http://lists.openstack.org/pipermail/openstack-dev/2018-November/136403.html
> > [2] 
> > http://specs.openstack.org/openstack-infra/infra-specs/specs/opendev-gerrit.html
> > [3] http://eavesdrop.openstack.org/#Project_Infrastructure_Team_Meeting
> > [4] https://ethercalc.openstack.org/opendev-transition
> > 
> >
> 
> We've made good progress in preparing this change and are still on 
> track to do this Friday April 19, 2019.  Our project liasons have been 
> drawing up lists of projects to rename during the outage. One big thing 
> to keep in mind is unofficial OpenStack projects will no longer be in 
> the "openstack" namespace.  They will be placed in the 'x/' namespace 
> instead which intends to indicate no endorsement or special ownership.
> 
> One side effect of this change (as noted in Jim's earlier email) is 
> that we will stop replicating projects that move out of the openstack 
> namespace. Projects that wish to be replicated to Github or anywhere 
> else they like can do so following the steps that David Moreau-Simard 
> put together for us here [5].
> 
> This transition is likely to be a bit bumpy particularly at the start. 
> We'll be around after the transition to help fix unexpected errors and 
> are likely to spend a fair bit of time at the PTG improving things as 
> well.
> 
> Finally, to be extra clear, we intend to put http redirects in place so 
> that all your old http(s) urls continue to work.  Fungi has set this up 
> for testing with details here [6] if you would like to ensure your urls 
> redirect properly.
> 
> [5] 
> http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html
> [6] 
> http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004921.html

The infra/OpenDev team continues to make good progress towards this transition and plans to perform the transition on April 19, 2019 as previously scheduled.  We will begin the transition at 15:00UTC and users should plan for intermittent Gerrit and git repo outages through the day. We expect most of those will be closer to 15:00UTC than 23:00UTC.

Fungi has generated a master list of project renames for the openstack namespaces: http://paste.openstack.org/show/749402/. If you have a moment please quickly review these planned renames for any obvious errors or issues.

For the airship, starlingx, and zuul repo renames the repositories listed at git.airshipit.org, git.starlingx.io, and git.zuul-ci.org were used placing repos in airship/, starlingx/ and zuul/ namespaces. Any repo name prefix (like stx- and airship-) is dropped.

As always we are happy to answer any questions you might have or address any concerns. Feel free to reach out.
Thank you for your patience,
Clark