Planet Ubuntu
Subscribe to Planet Ubuntu feed
Planet Ubuntu - http://planet.ubuntu.com/
Updated: 4 hours 6 min ago

Jorge Castro: Juju is now on Github

Wed, 2014-06-04 13:55

We’ve got some changes in Juju and the Juju ecosystem that have been landing this week.

Ian Booth announced the move of Juju core to github.com. You can find all our work at: https://github.com/juju.

Workflow instructions for contributing are available in the CONTRIBUTING file. Ian also adds:

Once the dust settles on the migration of juju-core, we’ll also be migrating various dependencies like goose, gwacl, gomaasapi and golxc.

You can find the code for Juju Core at: https://github.com/juju/juju

On a related note, we have a one way mirror of the Juju Charm Store as well: https://github.com/charms

You can combine these with Francesco Banconi’s git-deploy plugin to deploy right from github, as an example:

juju git-deploy charms/mysql

Hopefully 2-way syncing will be possible soon, stay tuned!

David Murphy: Enabling Students in a Digital Age: Charlie Reisinger at TEDxLancaster

Wed, 2014-06-04 13:44

This is really inspiring to me, on several levels: as an Ubuntu member, as a Canonical, and as a school governor.

Not only are they deploying Ubuntu and other open-source software to their students, they are encouraging those students to tinker with their laptops, and – better yet – some of those same students are directly involved in the development, distribution, and providing support for their peers. All of those students will take incredibly valuable experience with them into their future careers.

Well done.

The post Enabling Students in a Digital Age: Charlie Reisinger at TEDxLancaster appeared first on David Murphy.

David Tomaschik: Secuinside Quals 2014: Simple Login

Wed, 2014-06-04 02:08

In this challenge, we received the source for a site with a pretty basic login functionality. Aside from some boring forms, javascript, and css, we have this PHP library for handling the session management:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50<? class common{ public function getidx($id){ $id = mysql_real_escape_string($id); $info = mysql_fetch_array(mysql_query("select idx from member where id='".$id."'")); return $info[0]; } public function getpasswd($id){ $id = mysql_real_escape_string($id); $info = mysql_fetch_array(mysql_query("select password from member where id='".$id."'")); return $info[0]; } public function islogin(){ if( preg_match("/[^0-9A-Za-z]/", $_COOKIE['user_name']) ){ exit("cannot be used Special character"); } if( $_COOKIE['user_name'] == "admin" ) return 0; $salt = file_get_contents("../../long_salt.txt"); if( hash('crc32',$salt.'|'.(int)$_COOKIE['login_time'].'|'.$_COOKIE['user_name']) == $_COOKIE['hash'] ){ return 1; } return 0; } public function autologin(){ } public function isadmin(){ if( $this->getidx($_COOKIE['user_name']) == 1){ return 1; } return 0; } public function insertmember($id, $password){ $id = mysql_real_escape_string($id); mysql_query("insert into member(id, password) values('".$id."', '".$password."')") or die(); return 1; } } ?>

Some first impressions:

  • MySQL calls seem to be properly escaped.
  • The auth cookie is using the super-weak crc32.
  • Setting the user_name cookie to 'admin' won't work out for us.

In index.php, we see:

1 2 3if($common->islogin()){ if($common->isadmin()) $f = "Flag is : ".__FLAG__; else $f = "Hello, Guest!";

So, presumably, the correct user is actually 'admin', but we can't log in as that. So what to do? Well, after playing around for a bit, I realized one important point. By default, MySQL uses case-insensitive string comparisons but, of course, PHP's == operator is case-sensitive. So a mixed-case version of admin will pass the test in islogin() but will return the user we want in getidx(), but we can't log in as any variation of admin as the password will still be needed.

That brings us to the hash. Perhaps we could fake the hash for an uppercased admin user? While we could probably brute force the salt, that would take a while. However, crc32 is vulnerable to trivial hash length extension attacks, if you can set the internal state to an existing hash. That is: crc32(a+b) == crc32(b, crc32(a)). So, since the salt is at the beginning, if we have the crc32 for a user, we can easily concatenate anything on the end and still generate a valid hash. (Assuming an implementation of crc32 that allows you to set the existing internal state.)

One rub: while python allows you to set the state, it doesn't implement the same CRC-32 as PHP! (I thought there was only one CRC-32, but apparently the one in python's binascii and zlib modules is the zlib CRC-32, and the PHP hash one is the bz2 CRC-32.) So I was able to find the relevant lookup table for the BZ2 crc-32 and write this implementation:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18import struct crc_table = [ 0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L, ...snip... 0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L ] def bzcrc(s, init=None): if init: state = struct.unpack('>I', struct.pack('<I', ~init & 0xffffffff))[0] else: state = 0xffffffff for c in s: state = state & 0xffffffff state = ((state << 8) ^ (crc_table[(state >> 24) ^ (ord(c))])) return hex(struct.unpack('>I', struct.pack('<I', ~state & 0xffffffff))[0])

And yes, I do some weird stuff with byte-order swapping, but it works for the one off. So, we logged in as the user 'a', got a hash, then changed the user_name cookie to aDMIN, and calculated the new hash via: bzcrc('DMIN', <existing hash>). Updated the hash cookie, refresh, and we've got a flag.

Ubuntu Server blog: Meeting Minutes: June 3rd, 2014

Tue, 2014-06-03 19:27
Agenda
  • Review ACTION points from previous meeting
  • U Development
  • Server & Cloud Bugs (caribou)
  • Weekly Updates & Questions for the QA Team (psivaa)
  • Weekly Updates & Questions for the Kernel Team (smb, sforshee)
  • Ubuntu Server Team Events
  • Open Discussion
  • Announce next meeting date, time and chair
Minutes
  • vUDS is next week (Tues-Thurs) – Pat (gaughen) is still working on topics, so if someone has a suggestion please talk to her.
  • bug 1319555 should not be on the list – list needs refreshing
  • bug 1315052 has fix committed upstream
  • bug 1317587 is in progress
  • The team is working on getting the blueprints filled out completely.  Expecting them to be solidified around vUDS.
  • Louis (caribou) created blueprint: https://blueprints.launchpad.net/ubuntu/+spec/servercloud-u-networked-kdump and working on getting it filled in and approved.
  • kdump may be added to vUDS agenda
  • There’s an Openstack meetup in London on Thursday – James (jamespage) and Liam (gnuoy) are attending.  http://www.eventbooking.uk.com/openstack/home.html
Next Meeting

Next meeting will be on Tuesday, June 10th at 16:00 UTC in #ubuntu-meeting.

Additional logs @ https://wiki.ubuntu.com/MeetingLogs/Server/20140603

Ubuntu Kernel Team: Kernel Team Meeting Minutes – June 03, 2014

Tue, 2014-06-03 17:13
Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20140603 Meeting Agenda


ARM Status

No new update this week.


Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

http://people.canonical.com/~kernel/reports/kt-meeting.txt


Milestone Targeted Work Items    apw    core-1405-kernel    2 work items       ogasawara    core-1405-kernel    2 work items   


Status: Utopic Development Kernel

We have most recently rebased our Utopic kernel to v3.15-rc8 and
uploaded (3.15.0-5.10). We are planning on converging on the v3.16
kernel for Utopic. It also appears that the Utopic release date has
been pushed out a week to Thurs Oct 23 in order to not conflict with
the Linux Plumbers Conference.
—–
Important upcoming dates:
Mon-Wed June 10 – 12, UOS – Ubuntu Online Summit (~1 week away)
Thurs Jun 26 – Alpha 1 (~3 weeks away)
Fri Jun 27 – Kernel Freeze for 12.04.5 and 14.04.1 (~3 weeks away)


Status: CVE’s

The current CVE status can be reviewed at the following link:

http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html


Status: Stable, Security, and Bugfix Kernel Updates – Trusty/Saucy/Precise/Lucid

Status for the main kernels, until today (June 3):

  • Lucid – Verification and Testing
  • Precise – Verification and Testing
  • Quantal – No changes this cycle
  • Saucy – Verification and Testing
  • Trusty – Verification and Testing

    Current opened tracking bugs details:

  • http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

    For SRUs, SRU report is a good source of information:

  • http://people.canonical.com/~kernel/reports/sru-report.html

    Schedule:

    cycle: 18-May through 07-Jun
    ====================================================================
    16-May Last day for kernel commits for this cycle
    18-May – 24-May Kernel prep week.
    25-May – 31-May Bug verification & Regression testing.
    01-Jun – 07-Jun Regression testing & Release to -updates.


Open Discussion or Questions? Raise your hand to be recognized

No open discussions.

David Planella: A new era for the Ubuntu community team, or business as usual

Tue, 2014-06-03 17:06

A sample of the wider Ubuntu Community team, with Canonicalers and volunteer core app developers

After the recent news of Jono stepping down as the Ubuntu Community Manager to seek new challenges at XPRIZE, a new era in Ubuntu begins. Jono’s leadership, passion and drive to continually push the boundaries have been contagious over the years, and have been the catalyst for growing the unique community of individuals that defines Ubuntu today.

Jono is now joining the ranks of non-Canonical Ubuntu members, and while this will change the angle of participation, I’m certain that it won’t change his energy and dedication one bit. But most importantly, it’s a testament to his work that his former team will continue to thrive and take up the torch in pushing those boundaries.

For us, it will be business as usual in the sense of implementing our roadmap, continuing to grow a strong and open community, being innovative in how we do it, and coordinating the logistics around our plans. So not much will be different in that regard, but obviously some organizational bits will change.

As part of the transition, the Ubuntu Community Team at Canonical in full, that is, Michael Hall, Daniel Holbach, Alan Pope, Nicholas Skaggs and myself, will now be hosting the weekly Ubuntu Q&A, starting today at 18:00 UTC on Ubuntu On Air (click here for the time at your location).

The Ubuntu Community Team Q&A

Openness, both in being a transparent and welcoming community, is one of the core values of Ubuntu, and we believe the channels should be always open for a healthy information flow and to help contributors get involved.

As such, the Ubuntu Community Team Q&A will continue to provide a weekly, 1-hour-long session open for participation to anyone who wants to ask their questions about Ubuntu. In fact, as in former editions, you can ask the Community Team just anything about Free Software, Technology, or whatever you come up with. As before, the only questions we won’t answer are those related to technical support, where you’ll be much better served using Ask Ubuntu, the Ubuntu forums or IRC.

Join the Ubuntu Community Team Q&A at 18:00 UTC today and ask your questions >

The Ubuntu Online Summit is coming soon!

Also, following the thread of events and participation, the new Ubuntu Online Summit (UOS) is coming up very soon, and it’s an excellent opportunity to learn about getting involved in Ubuntu, organizing or presenting the plans of the different Ubuntu teams for the next months.

UOS will be held on June 10th – 12th and it will be a combination of the former Ubuntu Developer Summit and the more user-facing events we’ve been organizing in the past. This opens the door to a wider audience that can follow a richer mix of developer and user or contributor content.

If you want to learn about the details, check out Michael’s UOS post on how it’s going to work. If you want to contribute and make a difference in Ubuntu, do register a session too!

Looking forward to seeing you soon!

The post A new era for the Ubuntu community team, or business as usual appeared first on David Planella.

Svetlana Belkin: Calling for Community UOS 14.06 Tracks

Tue, 2014-06-03 14:09

The Ubuntu Online Submit is next week (June 12 – June 14) and we are still seeking proposals for all of the tracks that are listed in this blog post.  Since I’m one of the Community Track leads, you may ask me questions on how to propose a session/track or any other questions.  You can also suggest ideas to me and I can help you get them into a session/track.  Scheduling questions can be directed to me also.

See you at the UOS!

 

 

 


Daniel Pocock: Click to dial for mobile users of your web sites

Tue, 2014-06-03 09:47

If there was a trivial way to let mobile phone users call you from your web site, just by adding a single HTML element to the page, would you do it?

In fact, there is. It doesn't even require a mobile WebRTC browser. It works for virtually any smartphone and a growing number of desktops too.

Introducing the tel: URI

The tel: URI is defined in RFC 3966.

For most mobile phone users, if they click a link to a tel: URI, their browser will copy the link into their dialer for convenience.

To protect users against calls to 0900 premium rate numbers, the user still has to make one more click to confirm they want to dial.

Examples

Here is a tel: URI:

tel:+44-20-7135-7070

Here is how to create a link with it:

<a href="tel:+44-20-7135-7070">020 7135 7070 (from abroad: +44 20 7135 7070)</a>

and here is how it looks on the page:

Call me on 020 7135 7070 (from abroad: +44 20 7135 7070)

and here is what appears on the mobile device after a user clicks the tel: URI link:

For desktop users too

Many desktop users can also benefit from tel: URIs. If they have a modern telephone system in their office, the system administrator may have already added a tel: URI handler to their desktop.

Anyone with a software PBX or a SIP account can also potentially use the TBDialOut extension for Firefox to help convert tel: URIs into sip: URIs or URLs for some bespoke dialer.

For those who want extra convenience, the Telify extension for Firefox will look for phone numbers in any HTML page and display them as tel: URIs so you can click them even if the web developer overlooked this.

Nathan Haines: Ubuntu Installfest with OCLUG

Tue, 2014-06-03 03:35

Last Saturday, Ubuntu held an installfest along with the Orange County Linux Users Group (OCLUG) in Fullerton, California. Thanks to the enthusiasm of OCLUG and its members, and the assistance of volunteers from the Ubuntu California Local Community Team, the event was a success.

OCLUG used to hold Linux installfests all the time, but has been fairly dormant the past couple of years, with meeting attendance small but consistent. Late last year, they considered holding an installfest as a way to get more interest from students and the community. The LUG agreed that it was best to promote a single distribution to reduce confusion and that teasing or jokes about other software—even though good-natured—was to be avoided during the event. A simple majority agreed that a default Ubuntu install was the best distro to offer to new users and it was agreed that anyone who came in wanting to install specific software would be welcomed as well. This was a compromise that everyone was happy with and it allowed the installfest to be a focused event.

OCLUG meets once a month at California State University Fullerton, and so advertising for the event was done with flyers, which were posted around the campus and in nearby coffee shops. It contained a simple pitch for Ubuntu, a URL for OCLUG and a QR code for the OCLUG installfest information page. We also emailed school faculty with information about the installfest, attaching a PDF of the flyers as well as a single-page “talking points” flyer that had a bulleted list talking about Ubuntu, installfests, and OCLUG, to encourage faculty to discuss the event with their students.

Ubuntu California supplied their secondary banner and table cloth, and Canonical arranged for reimbursement for pizza costs. Both were funded via the Ubuntu community donations from the Ubuntu download page, so I am very grateful to the generosity of the community. Canonical also provided Ubuntu 14.04 LTS discs and a conference pack with giveaway items. I designed name badges for both the OCLUG volunteers and the installfest attendees, and I also adapted the installfest liability release forms and data sheet forms from the Installfest HOWTO so that they matched the flyers and other documents.

When the day of the installfest finally arrived, we had four Ubuntu volunteers and nine OCLUG volunteers. We had 7 attendees, with 4 who brought their computers for an install and 3 more who simply wanted to attend and learn more about Ubuntu. Everyone arrived on time and enjoyed the donuts and coffee provided by OCLUG as is usual for their meetings. We had a greeter or two by the parking structure to direct attendees to the classroom. An OCLUG volunteer passed out the installfest forms and I had the Ubuntu volunteers distribute a standard swag pack for each attendee: Ubuntu lanyard, sticker sheet, pen, button, and Desktop install disc. Stephan Ingram, the president of OCLUG welcomed everyone and introduced me, then I gave my presentation to the group. I briefly discussed operating systems and the ideals of Free Software so that I could go into detail about what and why Ubuntu offers a complete computing solution that is elegant and easy to use. I described the Ubuntu and local Linux communities, and then quickly explained the release forms and discussed some USB keys that were available for purchase. Then installation began.

Everyone helped out and the attendees were able to get Ubuntu installed on their machines and have conversations about computer and software, and everyone had a good time. I burned some 32-bit Ubuntu discs for a couple attendees and passed out the Xubuntu discs I had prepared for slower machines. The pizza came, and while everyone was eating I showed off Ubuntu on my phone, demonstrating phone and desktop convergence using the Weather app. After the pizza was finished, I returned to the front of the room to demonstrate the key features of the Unity desktop interface, discuss the benefits of Unity’s online search and how to turn it off, and how to enable Autohide, change the desktop background, and use the Ubuntu Software Center and the Unity Dash to search for and install applications. Using Stellarium as an example, I then proceeded to launch and demonstrate this virtual planetarium software as an example of the rich content available with Free Software solutions.

We ended the installfest with a giveaway. I drew names of attendees and we gave away an Android tablet and an external phone/tablet battery provided by OCLUG members, and then we gave away three exclusive Ubuntu Cloud t-shirts provided by Canonical in their conference pack. By the time the installfest was over, we had installed Ubuntu successfully on every target machine, passed out 35 Ubuntu Desktop discs and 3 Ubuntu Server discs, sold 5 USB drives, and impressed a faculty member who promised to promote the next installfest to his students because he said there was no reason they should have to pay for scientific software if they could have high quality software for free. He also discussed academic year timing with the OCLUG president and based on that there are preliminary plans to repeat the installfest in September when we should be able to attract more students.

Looking back, the flyers were designed for on-campus use but traveled further, so they should give a little more location context, and the installfest page should probably include specific event information instead of relying on the OCLUG main page. We only had a 4-hour window for the event and I still feel this isn’t quite long enough. I didn’t have much time to dedicate to the Ubuntu volunteers, all of whom were volunteering for the first time and while I felt bad about this, they all stepped up and excelled in a way that made me very proud. For September, I intend to engage the university’s radio, television, and newspapers to help spread the word a bit further on campus.

Photos of the event are available to download at http://people.ubuntu.com/~nhaines/images/events/2014/oc-installfest-may/

I’d like to encourage anyone in the Ubuntu community to modify and adapt any printable resource that would be helpful to them. All printable media as well as the source documents, the main presentation, and sanitized attendee records are available to download at http://people.ubuntu.com/~nhaines/documents/events/2014/oc-installfest-may/

The Fridge: Ubuntu Weekly Newsletter Issue 370

Mon, 2014-06-02 23:40

Welcome to the Ubuntu Weekly Newsletter. This is issue #370 for the week May 26 – June 1, 2014, and the full version is available here.

In this issue we cover:

The issue of The Ubuntu Weekly Newsletter is brought to you by:

  • Elizabeth K. Joseph
  • Paul White
  • Emily Gonyer
  • And many others

If you have a story idea for the Weekly Newsletter, join the Ubuntu News Team mailing list and submit it. Ideas can also be added to the wiki!

Except where otherwise noted, content in this issue is licensed under a Creative Commons Attribution 3.0 License BY SA Creative Commons License

Raphaël Hertzog: My Free Software Activities since January 2014

Mon, 2014-06-02 22:36

If you follow my blog closely, you noticed that I skipped all my usual monthly summaries in 2014. It’s not that I stopped doing free software work, instead I was just too busy to be able to report about what I did. As an excuse, let me tell you that we just moved into a new house which was in construction since may last year.

The lack of visible activity on my blog resulted in a steady decrease of the amount of donations received (January: 70.72 €, February: 71.75 €, March: 51.25 €, April: 39.9 €, May: 40.33 €). Special thanks to all the people who kept supporting my work even though I stopped reporting about it.

So let’s fix this. This report will be a bit less detailed since it covers the whole period since the start of the year.

Debian France

Preparations related to general assemblies. The year started with lots of work related to Debian France. First I took care of setting up limesurvey with Alexandre Delanoë to handle the vote to pick our new logo:

I also helped Sylvestre Ledru to finalize and close the accounting books for 2013 in preparation for the general assembly that was due later in the month. I wrote the moral report of the president to be presented to the assembly. And last step, I collected vote mandates to ensure that we were going to meet the quorum for the extraordinary assembly that was planned just after the usual yearly assembly.

The assemblies took place during a two days mini-debconf in Paris (January 17-18) where I was obviously present even though I gave no talk besides announcing the logo contest winner and thanking people for their participation.

The Debian France members during the general assembly

It’s worth noting that the extraordinary assembly was meant primarily to enshrine in our bylaws the possibility to act as a trusted organization for Debian. This status should be officialized by the Debian project leader (Lucas Nussbaum) in the upcoming weeks since we answered satisfactorily to all questions. Our paypal donation form and the accounting tools behind it are ready.

Galette packaging and members map. I managed to hand over the package maintenance of galette to François-Régis Vuillemin. I sponsored all his uploads and we packaged a new plugin that allows to create a map with all the members who accept to share their location. The idea was to let people meet each other when they don’t live far away… with the long term goal to have Debian France organized activities not only in Paris but everywhere in France.

New contributor game. Last but not least, I organized a game to encourage people to do their first contribution to Debian by offering them a copy of my book if they managed to complete a small Debian project. We got many interesting projects but the result so far seem to be very mixed. Many people did not complete their project (yet)… that said for the few that did substantial work, it was rather good and they seem to be interested to continue to contribute.

Debian France booth at Solutions Linux in Paris. Like each year, I spent two days in Paris to help man the Debian France booth at Solutions Linux. We had lots of goodies on sale and we made more than 2000 EUR in earnings during the two days. I also used this opportunity to try to convince companies to support the new Debian LTS effort.

Tanguy Ortolo and Fernando Lagrange behind the Debian France booth

The Debian Administrator’s Handbook

In the last days of 2013, we released the wheezy update of the book. Then I quickly organized everything needed so that the various translation teams can now focus their efforts on the latest release of the book.

Later (in February) I announced the availability of the French and Spanish translations.

Debian Squeeze LTS

When the security team called for help to try to put in place long term support for Squeeze, I replied positively because I’m convinced that it’s very important if Debian wants to stay an acceptable choice in big deployments and because I knew that some of my customers would be interested…

Thus I followed all the discussions (on a semi-private list first and then on debian-lts@lists.debian.org) and contributed my own experience. I have also taken up the responsibility to coordinate with the Debian contributors who can be hired to work on Squeeze LTS so that we have a clear common offer for all the companies who have offered financial support towards Squeeze LTS. Expect further news on this front in the upcoming days/weeks.

Tryton

I have been a long time user of SQL-Ledger to manage the accounting of my company Freexian. But while the license is free software, the project is not. It’s the work of a single developer who doesn’t really accept help. I have thus been considering to move to something else for a long time but never did anything.

This year, after some rough evaluation, I decided to switch to Tryton for my company. It’s probably not a wise choice from a business perspective because that migration took me many hours of unpaid labor but from a free software perspective it’s definitely better than everything else I saw.

I contributed a lot of bug reports and a few patches already (#3596, #3631, #3633, #3665, #3667, #3694, #3695, #3696, #3697) mainly about problems found in the French chart of accounts but also about missing features for my use case.

I also accepted to sponsor Matthias Berhle, who is maintaining the official Debian packages of Tryton. He’s already a Debian maintainer so it’s mainly a matter of reviewing new source packages and granting him the required rights.

Misc Debian work
  • Updated publican to version 4 and then 4.1.2. Required a new perl module that I requested to the Perl team in
    #736816.
  • Updated to python-django-debug-toolbar and python-django-jsonfield for Django 1.6 compatibility.
  • Filed bugs on packages depending against linux-image that got dropped (on request of Ben Hutchings)
  • Filed #734866 and #734869 against bash/dash to request that they properly drop privileges in setuid context.
  • Updated gnome-shell-timer.
  • Created “Services” pages on the wiki for the PTS and its replacement.
  • Worked on distro-tracker together with the participants of the new contributor game.
  • Orphaned feed2omb with #742601.
  • Tried in vain to fight against silliness of Debian specific changes in syslinux (see #742836).
  • Preliminary EFI support in live-build (see #731709).
  • Updated python-django to 1.6.5 in unstable, 1.4.5+deb7u7 in wheezy-security and 1.6.5-1~bpo70+1 to wheezy-backports.
  • Sponsored dolibarr, python-suds, a zim backport, a ckeditor NMU to fix an RC bug, libapache2-mod-form, ledgersmb.
  • Filed bugs on the fly: #749332 (new upstream release of libjs-jquery-cookie), #749498 (problem with Files-Excluded and https URL for copyright-format 1.0), #747354 (bug in clamav-milter init script), #747101 (git-import-orig should offer a –download option).
  • Filed tickets on mirrorbrain to make it work better with Debian mirrors: update to #26 (avoid error 404 on files still available on some mirrors) and #150 (auto-disable outdated mirrors).
Thanks

See you next month for a new summary of my activities.

No comment | Liked this article? Click here. | My blog is Flattr-enabled.

Simos Xenitellis: How to count the population of flocks of birds using software

Mon, 2014-06-02 22:22

We normally see around a few birds and it is rather easy to count them. However, when there is a flock with dozens, hundreds or thousands of birds, there is need for an automated (computer-assisted) method to count the birds. Without a computer-assisted method, it is easy to over/under-estimate the count, and worst of all, there is no way to provide evidence of your count to a third-party.

There are several efforts to use software to count birds and other wildlife. Juan M. Pérez-García in The use of digital photography in censuses of large concentrations of passerines: the case of a winter starling roost-site (2010) describes how to use the UHTSCSA Image Tool 3.0 (specialized image processing software primarily for medical applications) in order to count the population of flocks. He calculates the error from the automated counting from the images. The specific software is closed-source, for Windows only and was last updated in 2002.

Another option that is suitable for bigger birds such as flamingos is described at An automatic counter for aerial images of aggregations of large birds (2011) (full text) by Arnaud Béchet et al.  A specialized program was written, called FLAMINGO (distributed under the CeCILL license, a French free and open-source software license), that can count populations of birds that their body shape is shown as an ellipse in aerial photographs.

Sample aerial image showing a flock of flamingos.

Sample aerial image showing a flock of identified flamingos. Notice the tiny black dot on each bright area.

Each bright spot most likely corresponds to a bird, and the software manages to identify and count them. Apparently, the software has been tuned to work with such low resolution images, since the airplane probably had to fly high enough in order not to disturb the flock.

Another option to count birds is DotCount by Martin Reuter . DotCount is closed-source, available for Windows and OS/X.

Estimating Starling Flock Size, about 800 birds according to DotCount (source: DotCount Sample page)

I could not get it to work reliably with my photos. It works better with pre-processed images at low resolution.

A final option is to use ImageJ, as explained by Christof, at How to count the birds in a photo of a flock – automatically! I will be repeating the instructions here, with emphasis on how to use on Ubuntu. ImageJ has been developed in Java, thus it is also available in Windows and OS/X. Check at ImageJ on how to download and install on Windows or OS/X (then, continue at step 3 below).

  1. First of all, ImageJ is available at the Software Centre in Ubuntu. Find it and install it,

    ImageJ at the Software Centre in Ubuntu

    You will notice that the icon of ImageJ will then be automatically added to the launcher.

    ImageJ icon on the launcher in Ubuntu

    The icon looks like a golden optical microscope.

  2. The version that we just installed was 1.47a, and was released in the summer of 2012. At http://imagej.nih.gov/ij/notes.html we can see that there is a more recent version. Let's update! We need to close the ImageJ application (if it is running) and then open the Terminal window. There, we run gksudo imagej   which will run ImageJ with superuser privileges. The reason why we do this, is to click on the Help → Updated ImageJ... menu, and get ImageJ to auto-update itself. Do that. Once the update is completed, we can close ImageJ and start it again from the Launcher and get the latest ImageJ!
  3. Let's start with an example. Let's count the birds at

    Image of starlings, source: http://reuter.mit.edu/software/dotcount/examples

    Right-click on the image above and save it locally in order to open with ImageJ. Then, start ImageJ and open the image.

  4. In ImageJ, click on the menu Image → Type → 8-bit  in order to convert the image to an 8-bit image. This step simplifies the image for the next step. The colors are reduced as they are not important for our counting.
  5. Then, click on the menu Image → Adjust → Threshold... With the Threshold tool we can select what information on the image to keep, and easily remove the rest. We set lower and upper thresholds, and at the same time can see in the image when the birds and just the birds appear in the special red color. Once we are happy with the threshold values, we click on Apply. By applying, the features in the image that are in red will remain, and the rest are gone.
  6. Now let's count. Click on Analyze → Analyze Particles... (see documentation at http://rsbweb.nih.gov/ij/docs/guide/146-30.html)

    ImageJ - Analyze Particles tool

    In the Size text box we can specify the range for the size of each individual bird. 0-50 means that a single pixel up to a group of 50x50 (2500) pixels will be counted as an individual bird.  Circularity describes how circular the shapes of the birds should be. Circularity 1 means a perfect circle and 0 means not a circle at all. Therefore, the value 0.00-1.00 means that we accept here any shape. Finally, we click OK in order to perform the analysis.

  7. Here is the output,

    Starlings counted, summary

    Starlings counted, details

    You can notice that the branch/stick on the upper-left of the image has not been counted as a bird due to the upper size limit that we specified. The total number of birds has been counted to 803. Obviously there should be some errors, and depending on the set of images that we work on, it is important to calculate how big that error can be (comparing with hand-counting from the photograph, etc).

Let's try with another image, a photograph straight from the camera.

Birds in the sky (original)

You need to click on the image  in order to get the full resolution (4912 x 2760). Them, click to save as an image file.

You can distinguish most of the birds at the lower-right of the image. Feel free to try to count them manually before getting the result from ImageJ.

It is helpful, after you convert the image to 8-bit, to try to subtract the background of the image. For this image, the clouds will get dull which will help with the next steps in the processing. To subtract the background, click on Process→Substract background...

Below is an animated GIF that shows a section of the original photograph, in three animated frames. The first frame is the original photograph, after the background has been subtracted. The second frame shows the birds being identified after the threshold has been applied. The third frame has the birds numbered.

You may notice that two birds have not been picked up by ImageJ as they can barely be distinguished. Another source of error is when two or more birds overlap, and are counted as one (this case is not shown in the animated GIF). These two issues contribute to the margin of error, when using ImageJ.

Birds flying in the sky (cropped)

ImageJ can accept plugins, so it is feasible to write a BirdCounting plugin that performs these steps in one go.

Bird counting is a difficult task. By taking photographs of a flock, it is possible to count them and provide evidence of the count.

No comment

Related posts:

  1. Gangnam Style in Ubuntu
  2. Ubuntu usage stats with getclicky
  3. My Akismet spam count in WordPress

Pages