DNF into C initiative started

As you already know, so far DNF has been using a bunch of C libraries (hawkey, librepo, libsolv, libcomps) while yum was written entirely in Python. From now some of the DNF code will be slowly rewritten into C, more precisely, moved into libhif project. The next milestone was reached by merging hawkey into libhif and further we plan to expand libhif to support general functionality of package managers.

Why have we merged hawkey into libhif

Nowadays there are three major consumers of hawkey – DNF, PackageKit and rpm-ostree. The hawkey API was not in final form yet and was changed constantly based on demands from these package managers. We have merged hawkey project inside libhif and hidden some of not yet stable API.

Merging hawkey into libhif was another step to move more code base of DNF into C. DNF will reuse some of the existing code of libhif. Having this shared library can eliminate inconsistencies about installed packages when DNF and PackageKit is used alternately. Moreover we would like to reuse the same metadata for all package managers to save your bandwidth.

Libhif should contain the common functionality for all package managers. So far libhif is providing high level API by taking care of fetching metadata from mirrors, doing dependency solving and executing RPM transaction. In the future it will support repository configuration parsing, GPG checking and so on. At this time, this is handled by all package managers separately.

Facts for Hawkey consumers:

  • libhif-0.7.0 will obsolete hawkey package
  • some of the C hawkey API from libhif will not be exposed anymore, please use libhif functions instead
  • python bindings will not change and the libhif package will still provide python2-hawkey and python3-hawkey
  • API in libhif is still not considered as fully stable yet
  • first release of libhif with hawkey inside is targeted for Fedora 25

Please watch libhif project on github and participate in pull request discussions so you can influence the development.

18 thoughts on “DNF into C initiative started

    1. Relying the same message as to fedora devel list, hope it answers your question:

      Having DNF in C is desire of many users. If you could notice the thread
      from Courtney Pacheco, she did investigation [1] to minimize the Fedora
      image (which is bigger than in other distros). The DNF will probably be
      taken out of that image. Mostly because of the Python requirement.

      By dropping Python requirement, it could speed up the the process start-up
      (no need to load Python libs then). That is also the reason why PK does not
      call DNF Python API from C and rather have a special library in C instead.

      Gradually DNF is moving more code into C but still a lot of code is in
      Python and plugins are written in Python too. Having rewritten DNF into
      C would take a few years while maintaining the UX compatibility and
      still support Python API for plugins.

      The main message was to inform the consumers of hawkey to prepare their
      apps for the libhif change.

      [1] https://gist.github.com/iamcourtney/b8709ed897b7ecc9ac0f

    2. Hi Jan,

      Thanks a lot for this effort. Core parts of DNF should have never been written in Python. I spent substantial part of today converting DNF 1.1.8 to RHEL7, python versions and bindings being the main problem.

      After conversion to C all should be not only smaller dependency-wise but also faster.

      Don’t worry about non-glibc environments. All mainline distros are using glibc anyway. Non-GNU libc based ones already have much more troubles than this one with DNF…

      Thanks,
      Jindrich

    1. hawkey was glibc dependent a long time ago. By merging it into libhif we can use there glib too. Glib dependency was required from librepo and libcomps before so having hawkey without that would have no benefit as it is used mostly together with these libs. Was hawkey used in MUSL based distros?

      1. Can’t there be a next time? I think Fedora could be improved and optimized in order to be more streamlined without losing functionality.

        Maybe there can be some competition to Debian, ArchLinux and Gentoo distros in the embedded applications. Maybe Redhat can be wise and reduce distro size. Maybe they can provide the best of both binary and source based Linux distributions.

        Maybe Redhat can stop looking like a shady company and recover the equivalent the glory of Red Hat days.

        I seriously see DNF honors the name: So extremely long development cycle!

        Why are years required to make DNF full C? Why using Python and then C? Are there some sanity in Linux these days? I don’t see it.

        What about using LUA for scripting instead Python? LUA is a lot more lightweight.

        Can anyone explain this mess? I can’t! And why are two codebases at different Git repos?

        https://github.com/rpm-software-management/libcomps (Jul 2, 2015!)
        http://pkgs.fedoraproject.org/cgit/rpms/libcomps.git (Feb 4, 2016)

        https://github.com/rpm-software-management/dnf (Feb 26, 2016)
        http://pkgs.fedoraproject.org/cgit/rpms/dnf.git/ (Feb 25, 2016)

        http://pkgs.fedoraproject.org/cgit/rpms/dnfdaemon.git/ (Feb 3, 2016)

        http://pkgs.fedoraproject.org/cgit/rpms/dnf-langpacks.git/ (Feb 3, 2016)

        https://github.com/rpm-software-management/libhif (Feb 18, 2016)
        http://pkgs.fedoraproject.org/cgit/rpms/libhif.git/ (Feb 12, 2016)

        https://github.com/rpm-software-management/deltarpm (Jan 12, 2015!)
        http://pkgs.fedoraproject.org/cgit/rpms/deltarpm.git (Feb 3, 2016)

        https://github.com/rpm-software-management/rpm (Dec 3, 2015)
        http://pkgs.fedoraproject.org/cgit/rpms/rpm.git (Feb 29, 2016)

        https://github.com/rpm-software-management/yum-metadata-parser (Apr 5, 2013!)
        http://pkgs.fedoraproject.org/cgit/rpms/yum-metadata-parser.git (Feb 5, 2016)

        https://github.com/rpm-software-management/dnf-plugins-core (Feb 25, 2016)
        http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugins-core.git/ (Feb 25, 2016)

        https://github.com/rpm-software-management/dnf-plugin-system-upgrade (Dec 3, 2015)
        http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugin-system-upgrade.git/ (Feb 3, 2016)

        http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugin-spacewalk.git/ (Feb 3, 2016)

        https://github.com/rpm-software-management/dnf-plugins-extras (Dec 13, 2015)
        http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugins-extras.git/ (Feb 3, 2016)

        https://github.com/rpm-software-management/createrepo_c (Jan 5, 2016)
        http://pkgs.fedoraproject.org/cgit/rpms/createrepo_c.git (Feb 3, 2016)

        https://github.com/rpm-software-management/librepo (Feb 22, 2016)
        http://pkgs.fedoraproject.org/cgit/rpms/librepo.git (Feb 4, 2016)

        http://pkgs.fedoraproject.org/cgit/rpms/libsolv.git (Feb 27, 2016)

        1. Can anyone explain this mess? I can’t! And why are two codebases at different Git repos?

          https://github.com/rpm-software-management/libcomps (Jul 2, 2015!)
          http://pkgs.fedoraproject.org/cgit/rpms/libcomps.git (Feb 4, 2016)

          https://github.com/rpm-software-management/dnf (Feb 26, 2016)
          http://pkgs.fedoraproject.org/cgit/rpms/dnf.git/ (Feb 25, 2016)

          http://pkgs.fedoraproject.org/cgit/rpms/dnfdaemon.git/ (Feb 3, 2016)

          http://pkgs.fedoraproject.org/cgit/rpms/dnf-langpacks.git/ (Feb 3, 2016)

          https://github.com/rpm-software-management/libhif (Feb 18, 2016)
          http://pkgs.fedoraproject.org/cgit/rpms/libhif.git/ (Feb 12, 2016)

          https://github.com/rpm-software-management/deltarpm (Jan 12, 2015!)
          http://pkgs.fedoraproject.org/cgit/rpms/deltarpm.git (Feb 3, 2016)

          https://github.com/rpm-software-management/rpm (Dec 3, 2015)
          http://pkgs.fedoraproject.org/cgit/rpms/rpm.git (Feb 29, 2016)

          https://github.com/rpm-software-management/yum-metadata-parser (Apr 5, 2013!)
          http://pkgs.fedoraproject.org/cgit/rpms/yum-metadata-parser.git (Feb 5, 2016)

          https://github.com/rpm-software-management/dnf-plugins-core (Feb 25, 2016)
          http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugins-core.git/ (Feb 25, 2016)

          https://github.com/rpm-software-management/dnf-plugin-system-upgrade (Dec 3, 2015)
          http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugin-system-upgrade.git/ (Feb 3, 2016)

          http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugin-spacewalk.git/ (Feb 3, 2016)

          https://github.com/rpm-software-management/dnf-plugins-extras (Dec 13, 2015)
          http://pkgs.fedoraproject.org/cgit/rpms/dnf-plugins-extras.git/ (Feb 3, 2016)

          https://github.com/rpm-software-management/createrepo_c (Jan 5, 2016)
          http://pkgs.fedoraproject.org/cgit/rpms/createrepo_c.git (Feb 3, 2016)

          https://github.com/rpm-software-management/librepo (Feb 22, 2016)
          http://pkgs.fedoraproject.org/cgit/rpms/librepo.git (Feb 4, 2016)

          http://pkgs.fedoraproject.org/cgit/rpms/libsolv.git (Feb 27, 2016)

          This is not mess but upstream and downstream repositories ;-).

          1. Why having such complicated way? What about just one repo for all? I still don’t get it, sorry.

      2. Hawkey only needed a small change to work on non-glibc environments, and that change was applied to libhif[1].

        Hawkey uses glib, which can be built and used on non-glibc environments.

        Unity Linux is a distribution using musl[2] as its libc, and it is an RPM-based environment. It was previously using Yum, but it is moving to DNF[3].

        [1]: https://github.com/rpm-software-management/libhif/commit/615d2657a2eddfe2d6732b734632bf024db73a25

        [2]: http://www.musl-libc.org/

        [3]: https://gitlab.com/unity-linux-pkgs-core/dnf

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current day month ye@r *