[tc] [ironic] Promoting ironic to a top-level opendev project?
On Wed, Apr 15, 2020 at 9:55 PM Ben Nemec <openstack at nemebean.com> wrote:
> On 4/7/20 5:15 AM, Dmitry Tantsur wrote:
> > Hi,
> > On Mon, Apr 6, 2020 at 11:06 PM Ben Nemec <openstack at nemebean.com
> > <mailto:openstack at nemebean.com>> wrote:
> > At the risk of getting defensive, I do want to make some points
> > relating
> > to Oslo specifically here.
> > > 1) oslo.config is a fantastic piece of software that the whole
> > python
> > > world could benefit from. Same for oslo.service, probably.
> > oslo.config is an interesting case because there is definitely
> > outside OpenStack thanks to things like the Castellan config backend,
> > but as Doug mentioned oslo.config is heavily opinionated (global
> > object, anyone?) and that's an issue for a lot of people. I will also
> > point out that the only oslo.* dependency that oslo.config has is
> > oslo.i18n, which itself has no other oslo dependencies, so there's
> > much barrier to entry to using it standalone today.
> > oslo.config currently pulls in 16 dependencies to a clean venv,
> > including oslo.18n, Babel (why?) and requests (WHY?).
> Well, it's got translated strings and thus i18n, Babel is used for lazy
> translation, and requests is used for the remote file driver. It's
> posssible the latter two could/should be optional deps, but without them
> you do lose functionality.
Please pardon my ignorance, it's the first time I hear about this driver?
How many people are using it in the wild? Will it hurt much if we treat
this dependencies similar to database/tooz backend, i.e. as optional?
More on Babel below.
> > I would not inflict oslo.service on anyone I liked. :-P
> > So, you don't like us? :-P
> I don't _think_ I'm responsible for anyone using oslo.service. If I am,
> it happened years ago before I had dug into oslo.service much. ;-)
> > Seriously though, I would advocate for using cotyledon if you're
> > looking
> > for a general purpose service library. It's not eventlet-based and
> > provides a lot of the functionality of oslo.service, at least as I
> > understand it. It was also written by an ex-Oslo contributor outside
> > OpenStack so maybe it's an interesting case study for this
> > discussion. I
> > don't know how much contribution it gets from other sources, but that
> > would be good to look into.
> > Good to know, thanks!
> > > 2) oslo.utils as a catch-all repository of utilities should IMO be
> > > either moved to existing python projects or decomposed into small
> > > generally reusable libraries (essentially, each sub-module could
> > be a
> > > library of its own). Same for oslo.concurrency.
> > oslo.concurrency has already been decomposed into general purpose
> > (fasteners) and OpenStack-specific, at least for the most part. I'm
> > sure
> > the split isn't perfect, but it's not like we haven't recognized the
> > need for better concurrency libraries in Python as a whole. Note that
> > fasteners also lives outside Oslo in another ex-Osloers personal
> > Again, I'm not sure whether that was beneficial or not but it might
> > worth reaching out to Mehdi and Josh to see how they feel about it.
> > I *think* we mostly use oslo.concurrency for its execute() wrapper,
> > which seems like it could be a very useful library on its own (can I
> > suggest better-execute as a name?).
> Certainly something we could explore. Note that quite a few projects
> also use interprocess locks from o.c, in which case they're implicitly
> picking up some standard config opts for setting things like lock_path.
> I don't know if that's the case for Ironic though.
We only use in-process locks.
> > I would probably -2 any attempt to split oslo.utils. You'd end up
> > with a
> > bunch of single file libraries and a metric ****-ton of
> > overhead. It's bad enough managing 40 some repos as it is. Also,
> > another library with minimal cross-dependencies with the rest of Oslo
> > (just oslo.i18n again), which was an intentional design decision to
> > make
> > it relatively painless to pull in.
> > I guess I have a personal dislike of libraries without a simple goal
> > (I've been also fighting with various "utils" modules inside Ironic -
> > with only limited luck).
> I guess I should have noted that I agree "utils" and "misc" modules tend
> to be a code smell, but in this case we had two bad options available
> and had to pick one. IMHO we've done a reasonable job of keeping
> oslo.utils pretty minimal, but obviously I'm a bit biased. :-)
I guess I'm fine with that if we stop pulling in Babel :)
By the way, only oslo.i18n imports Babel directly. I'm proposing patches to
various projects that move it out of requirements.txt. If you see one -
E.g. a shameless ping for osc-lib folks: https://review.opendev.org/717737
> > I do suspect that at least some of oslo.utils contents could find a new
> > home, some maybe even in the stdlib. I also don't want us to end up in
> > the leafpad situation, on the other hand.
> That's entirely possible. We kind of lost our stdlib guy when Victor
> left the Oslo team so there may be things we've overlooked that could
> have been upstreamed.
> > > 3) I'm genuinely not sure why oslo.log and oslo.i18n exist and
> > > parts of their functionality cannot be moved upstream.
> > oslo.log basically provides convenience functions for OpenStack
> > services, which is kind of what the oslo.* libraries should do. It
> > provides built-in support for things like context objects, which are
> > fairly domain-specific and would be difficult to generalize. It also
> > provides a common set of configuration options so each project
> > have to write their own. We don't need 20 different ways to enable
> > debug
> > logging. :-)
> > oslo.i18n is mostly for lazy translation, which apparently is a thing
> > people still use even though the company pushing for it in the first
> > place no longer cares. We've also had calls to remove it completely
> > because it's finicky and tends to break things if the consuming
> > projects
> > don't follow the best practices with translated strings. So it's in a
> > weird limbo state.
> > May I join these calls please? :) Will I break Zanata (assuming anybody
> > still translates ironic projects) if I switch to the upstream gettext?
> I don't think you'll break Zanata, I think you'll break anyone who was
> using lazy translation. I know there are users of it out there, but
> maybe we should look into getting a user survey question for Oslo that
> would give us an idea of how widely its used.
I meant changing ironic, which honestly doesn't have real translations any
> > I did talk with JP in Shanghai about possibly making it optional
> > because
> > it pulls in a fair amount of translation files which can bloat
> > containers. I looked into it briefly and I think it would be
> > although I don't remember a lot of details because nobody was really
> > pushing for it anymore so it's hard to justify spending a bunch of
> > on it.
> > I'd be curious to hear more.
> Upon reflection, I think the thing we were looking to make optional was
> the Babel dep of oslo.i18n. It pulls in something like 6 MB of
> translation files
Oh, yeah. When building our ramdisk, we have to manually delete these files
to make it smaller.
> and is only used to look up the available languages
> for lazy translation. I think code changes were needed though, it wasn't
> as simple as moving Babel to optional. There's probably an argument that
> we shouldn't be making that call when lazy translation is disabled
> anyway though.
Is it something I can help with?
It seems that Babel is only used in get_available_languages. Can we make it
an optional dependency for projects that do use this call (ironic does not)?
> > Dmitry
> > TLDR: I think Oslo provides value both in and out of OpenStack
> > (shocking, I know!). I'm sure the separation isn't perfect, but what
> > There are some projects that have been split out of it that might be
> > interesting case studies for this discussion if anyone wants to
> > up. Not it. ;-)
> > -Ben
-------------- next part --------------
An HTML attachment was scrubbed...