LUG Community Blogs

Chris Lamb: Free software activities in August 2016

Planet ALUG - Wed, 31/08/2016 - 22:48

Here is my monthly update covering what I have been doing in the free software world (previously):

  • Worked on nsntrace, a userspace tool to perform network traces on processes using kernel namespaces:
    • Overhauled error handling to ensure the return code of the wrapped process is returned to the surrounding environment. (#10).
    • Permit the -u argument to also accept uids as well as usernames. (#16).
    • Always kill the (hard-looping) udp_send utility, even on test failures. (#13).
    • Updated configure.ac to look for iptables in /sbin & /usr/sbin (#11) and to raise an error if pcap.h is missing (#15).
    • Drop bashisms in #!/bin/sh script (#14) and ignore the generated manpage in the Git repository (#12).
  • Independently discovered an regression in the Django web development framework where field__isnull=False filters were not working with some foreign keys, resulting in extending the testsuite and release documentation. (#7104).
  • Proposed a change to django-enumfield (a custom field for type-safe constants) to ensure passing a string type to Enum.get returned None on error to match the documentation. (#36).
  • Fixed an issue in the Mopidy music player's podcast extension where the testsuite was failing tests in extreme timezones. (#40).
  • Proposed changes to make various upstream's reproducible:
    • botan, a crypto/TLS library for C++11. (#587).
    • cookiecutter, a project template generator, removing nondeterministic keyword arguments from appearing in the documentation. (#800).
    • pyicu, a Python wraper for the IBM Unicode library. (#27).
  • Integrated a number of issues raised by @piotr1212 to python-fadvise, my Python interface to posix_fadvise(2), where the API was not being applied to open file descriptors (#1) and moving the .so to a module directory (#2).
  • Various improvements to try.diffoscope.org, a hosted version of the diffoscope in-depth and content-aware diff utility, including introducing an HTTP API (#21), updating the SSL certificate and correcting a logic issue where errors in diffoscope itself were not being detected correctly (b0ff49). Continued thanks to Bytemark for sponsoring the hardware.
  • Fixed a bug in django-slack, my library to easily post messages to the Slack group-messaging utility, correcting an EncodeError exception under Python 3 (#53) and updated the minimum required version of Django to 1.7 (#54).
  • Various updates to tickle-me-email, my Getting Things Done-inspired email toolbox, to also match / in IMAP's LIST separators (#6) and to encode the folder list as UTF-7 (#7). Thanks to @resiak.
  • Clarified the documentation for travis.debian.net — my hosted script to easily test and build Debian packages on the Travis CI continuous integration platform — regarding how to integrate with Github (#20).

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most Linux distributions provide binary (or "compiled") packages to end users.

The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously and accidentally — during this compilation process by promising identical binary packages are always generated from a given source.


Toolchain issues

I submitted the following patches to fix reproducibility-related toolchain issues:


My work in the Reproducible Builds project was also covered in our weekly reports. (#67, #68, #69, #70).


Diffoscope

diffoscope is our "diff on steroids" that will not only recursively unpack archives but will transform binary formats into human-readable forms in order to compare them:

  • Added a command-line interface to the try.diffoscope.org web service.
  • Added a JSON comparator.
  • In the HTML output, highlight lines when hovering to make it easier to visually track.
  • Ensure that we pass str types to our Difference class, otherwise we can't be sure we can render them later.
  • Testsuite improvements:
    • Generate test coverage reports.
    • Add tests for Haskell and GitIndex comparators.
    • Completely refactored all of the comparator tests, extracting out commonly-used routines.
    • Confirm rendering of text and HTML presenters when checking non-existing files.
    • Dropped a squashfs test as it was simply too unreliable and/or has too many requirements to satisfy.
  • A large number of miscellaneous cleanups, including:
    • Reworking the comparator setup/preference internals by dynamically importing classes via a single list.
    • Split exceptions out into dedicated diffoscope.exc module.
    • Tidying the PROVIDERS dict in diffoscope/__init__.py.
    • Use html.escape over xml.sax.saxutils.escape, cgi.escape, etc.
    • Removing hard-coding of manual page targets names in debian/rules.
    • Specify all string format arguments as logging function parameters, not using interpolation.
    • Tidying imports, correcting indentation levels and drop unnecessary whitespace.

disorderfs

disorderfs is our FUSE filesystem that deliberately introduces nondeterminism in system calls such as readdir(3).

  • Added a testsuite to prevent regressions. (f124965)
  • Added a --sort-dirents=yes|no option for forcing deterministic ordering. (2aae325)

Other
  • Improved strip-nondeterminism, our tool to remove specific nondeterministic information after a build:
    • Match more styles of Java .properties files.
    • Remove hyphen from "non-determinism" and "non-deterministic" throughout package for consistency.
  • Improvements to our testing infrastucture:
    • Improve the top-level navigation so that we can always get back to "home" of a package.
    • Give expandable elements cursor: pointer CSS styling to highlight they are clickable.
    • Drop various trailing underlined whitespaces after links.
    • Explicitly log that build was successful or not.
    • Various code-quality improvements, including prefering str.format over concatentation.
  • Miscellaneous updates to our filter-packages internal tool:
    • Add --random=N and --url options.
    • Add support for --show=comments.
    • Correct ordering so that --show-version runs after --filter-ftbfs.
    • Rename --show-ftbfs to --filter-ftbfs and --show-version to --show=version.
  • Created a proof-of-concept reproducible-utils package to contain commonly-used snippets aimed at developers wishing to make their packages reproducible.


I also submitted 92 patches to fix specific reproducibility issues in advi, amora-server, apt-cacher-ng, ara, argyll, audiotools, bam, bedtools, binutils-m68hc1x, botan1.10, broccoli, congress, cookiecutter, dacs, dapl, dateutils, ddd, dicom3tools, dispcalgui, dnssec-trigger, echoping, eekboek, emacspeak, eyed3, fdroidserver, flashrom, fntsample, forkstat, gkrellm, gkrellm, gnunet-gtk, handbrake, hardinfo, ircd-irc2, ircd-ircu, jack-audio-connection-kit, jpy, kxmlgui, libbson, libdc0, libdevel-cover-perl, libfm, libpam-ldap, libquvi, librep, lilyterm, mozvoikko, mp4h, mp4v2, myghty, n2n, nagios-nrpe, nikwi, nmh, nsnake, openhackware, pd-pdstring, phpab, phpdox, phpldapadmin, pixelmed-codec, pleiades, pybit, pygtksourceview, pyicu, python-attrs, python-gflags, quvi, radare2, rc, rest2web, roaraudio, rt-extension-customfieldsonupdate, ruby-compass, ruby-pg, sheepdog, tf5, ttf-tiresias, ttf-tiresias, tuxpaint, tuxpaint-config, twitter-bootstrap3, udpcast, uhub, valknut, varnish, vips, vit, wims, winswitch, wmweather+ & xshisen.


Debian GNU/Linux Patches contributed

I also submitted 22 patches to fix typos in debian/rules files against ctsim, f2c, fonts-elusive-icons, ifrit, ldapscripts, libss7, libvmime, link-grammar, menulibre, mit-scheme, mugshot, nlopt, nunit, proftpd-mod-autohost, proftpd-mod-clamav, rabbyt, radvd, ruby-image-science, snmpsim, speech-tools, varscan & whatmaps.

Debian LTS

This month I have been paid to work 15 hours on Debian Long Term Support (LTS). In that time I did the following:

  • "Frontdesk" duties, triaging CVEs, etc.
  • Authored the patch & issued DLA 596-1 for extplorer, a web-based file manager, fixing an archive traversal exploit.
  • Issued DLA 598-1 for suckless-tools, fixing a segmentation fault in the slock screen locking tool.
  • Issued DLA 599-1 for cracklib2, a pro-active password checker library, fixing a stack-based buffer overflow when parsing large GECOS fields.
  • Improved the find-work internal tool adding optional colour highlighting and migrating it to Python 3.
  • Wrote an lts-missing-uploads tool to find mistakes where there was no correponding package in the archive after an announcement.
  • Added optional colour highlighting to the lts-cve-triage tool.
Uploads
  • redis 2:3.2.3-1 — New upstream release, move to the DEP-5 debian/copyright format, ensure that we are running as root in LSB initscripts and add a README.Source regarding our local copies of redis.conf and sentinel.conf.
  • python-django:
    • 1:1.10-1 — New upstream release.
    • 1:1.10-2 — Fix test failures due to mishandled upstream translation updates.

  • gunicorn:
    • 19.6.0-2 — Reload logrotate in the postrotate action to avoid processes writing to the old files and move to DEP-5 debian/copyright format.
    • 19.6.0-3 — Drop our /usr/sbin/gunicorn{,3}-debian and related Debian-specific machinery to be more like upstream.
    • 19.6.0-4 — Drop "template" systemd .service files and point towards examples and documentation instead.

  • adminer:
    • 4.2.5-1 — Take over package maintenance, completely overhauling the packaging with a new upstream version, move to virtual-mysql-server to support MariaDB, updating package names of dependencies and fix the outdated Apache configuration.
    • 4.2.5-2 — Correct the php5 package names.

Bugs filed (without patches) RC bugs

I filed 3 RC bugs with patches:



I additionally filed 8 RC bugs for packages that access the internet during build against autopkgtest, golang-github-xenolf-lego, pam-python, pexpect, python-certbot, python-glanceclient, python-pykka & python-tornado.



I also filed 74 FTBFS bugs against airlift-airline, airlift-slice, alter-sequence-alignment, apktool, atril, auto-apt-proxy, bookkeeper, bristol, btfs, caja-extensions, ccbuild, cinder, clustalo, colorhug-client, cpp-netlib, dimbl, edk2, elasticsearch, ganv, git-remote-hg, golang-codegangsta-cli, golang-goyaml, gr-radar, imagevis3d, jacktrip, jalv, kdepim, kiriki, konversation, libabw, libcereal, libdancer-plugin-database-perl, libdist-zilla-plugins-cjm-perl, libfreemarker-java, libgraph-writer-dsm-perl, libmail-gnupg-perl, libminc, libsmi, linthesia, lv2-c++-tools, lvtk, mate-power-manager, mcmcpack, mopidy-podcast, nageru, nfstrace, nova, nurpawiki, open-gram, php-crypt-gpg, picmi, projectl, pygpgme, python-apt, python-django-bootstrap-form, python-django-navtag, python-oslo.config, qmmp, qsapecng, r-cran-sem, rocs, ruby-mini-magick, seahorse-nautilus, shiro, snap, tcpcopy, tiledarray, triggerhappy, ucto, urdfdom, vmmlib, yara-python, yi & z3.


FTP Team

As a Debian FTP assistant I ACCEPTed 90 packages: android-platform-external-jsilver, android-platform-frameworks-data-binding, camlpdf, consolation, dfwinreg, diffoscope, django-restricted-resource, django-testproject, django-testscenarios, gitlab-ci-multi-runner, gnome-shell-extension-taskbar, golang-github-flynn-archive-go-shlex, golang-github-jamesclonk-vultr, golang-github-weppos-dnsimple-go, golang-golang-x-time, google-android-ndk-installer, haskell-expiring-cache-map, haskell-hclip, haskell-hdbc-session, haskell-microlens-ghc, haskell-names-th, haskell-persistable-record, haskell-should-not-typecheck, haskell-soap, haskell-soap-tls, haskell-th-reify-compat, haskell-with-location, haskell-wreq, kbtin, libclipboard-perl, libgtk3-simplelist-perl, libjs-jquery-selectize.js, liblemon, libplack-middleware-header-perl, libreoffice, libreswan, libtest-deep-json-perl, libtest-timer-perl, linux, linux-signed, live-tasks, llvm-toolchain-3.8, llvm-toolchain-snapshot, lua-luv, lua-torch-image, lua-torch-nn, magic-wormhole, mini-buildd, ncbi-vdb, node-ast-util, node-es6-module-transpiler, node-es6-promise, node-inline-source-map, node-number-is-nan, node-object-assign, nvidia-graphics-drivers, openhft-chronicle-bytes, openhft-chronicle-core, openhft-chronicle-network, openhft-chronicle-threads, openhft-chronicle-wire, pycodestyle, python-aptly, python-atomicwrites, python-click-log, python-django-casclient, python-git-os-job, python-hypothesis, python-nosehtmloutput, python-overpy, python-parsel, python-prov, python-py, python-schema, python-tackerclient, python-tornado, pyvo, r-cran-cairo, r-cran-mi, r-cran-rcppgsl, r-cran-sem, ruby-curses, ruby-fog-rackspace, ruby-mixlib-archive, ruby-tzinfo-data, salt-formula-swift, scapy3k, self-destructing-cookies, trollius-redis & websploit.

Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Wed, 31/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Tue, 30/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Mon, 29/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Sun, 28/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Sat, 27/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Fri, 26/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Thu, 25/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Wed, 24/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Tue, 23/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Anton Piatek: Now with added SSL from letsencrypt

Planet HantsLUG - Mon, 22/08/2016 - 19:51

I’ve had SSL available on my site for some time using startssl, but as the certificate was expiring and requires manual renewal, I though it was time to try out letsencrypt. I’m a huge fan of the idea of letsencrypt, which is trying to bring free SSL encryption to the whole of the internet, in particular all the smaller sites who might not have the expertise to roll out SSL or where a cost might be restrictive.

There are a lot of scripts for powering letsencrypt, but getssl looked the best fit for my use case as I just wanted a simple script to generate certificates, not manage apache configs or anything else. It seems to do a pretty good job so far. I swapped over the certificates to the newly generated ones and it seems pretty smooth sailing.

Categories: LUG Community Blogs

Now with added SSL from letsencrypt

Planet SurreyLUG - Mon, 22/08/2016 - 19:51

I’ve had SSL available on my site for some time using startssl, but as the certificate was expiring and requires manual renewal, I though it was time to try out letsencrypt. I’m a huge fan of the idea of letsencrypt, which is trying to bring free SSL encryption to the whole of the internet, in particular all the smaller sites who might not have the expertise to roll out SSL or where a cost might be restrictive.

There are a lot of scripts for powering letsencrypt, but getssl looked the best fit for my use case as I just wanted a simple script to generate certificates, not manage apache configs or anything else. It seems to do a pretty good job so far. I swapped over the certificates to the newly generated ones and it seems pretty smooth sailing.

Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Mon, 22/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Sun, 21/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Sat, 20/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Fri, 19/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Thu, 18/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Wed, 17/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Steve Kemp: This blog has moved

Planet HantsLUG - Tue, 16/08/2016 - 19:30
This blog has moved to https://blog.steve.fi/. Please update to use the new feed location.
Categories: LUG Community Blogs

Debian Bits: Debian turns 23!

Planet HantsLUG - Tue, 16/08/2016 - 13:30

Today is Debian's 23rd anniversary. If you are close to any of the cities celebrating Debian Day 2016, you're very welcome to join the party!

If not, there's still time for you to organize a little celebration or contribution to Debian. For example, you can have a look at the Debian timeline and learn about the history of the project. If you notice that some piece of information is still missing, feel free to add it to the timeline.

Or you can scratch your creative itch and suggest a wallpaper to be part of the artwork for the next release.

Our favorite operating system is the result of all the work we have done together. Thanks to everybody who has contributed in these 23 years, and happy birthday Debian!

Categories: LUG Community Blogs
Syndicate content