Search

GNU wget: Get all the web content you like on your local machine

October 17th, 2007 edited by Patrick Murena

Article submitted by Chris Camacho. We are running out of articles! Please help DPOTD and submit good articles about software you like!

Wget is so flexible you’ve probably been using it for years without knowing it, many scripts use it because it is a boilerplate method of grabbing files, it will even automatically retry under certain circumstances…

Probably the best compliment I can pay to it, is that script writers can use it and then forget about it.

It’s one of those great tools that makes *nix so great, its simple, it does what it says on the tin, and like many other pieces of unix “glue”, it’s robust.

Another cool thing about wget is that it’s non-interactive, which means you can start a new download, disconnect from your current session and find your downloads the next time you connect again.

How to use it

The most simple way to invoke wget is by typing wget URL/fileName

$ wget https://debaday.debian.net/contribute/

If you typed in this command you now have a file called index.html in the directory you where in while typing. This file contains the contribute page of the Debian package of the day blog. Read It, DPOTD needs you ;)

Get a directory hierarchy

To get the full content of a directory and his subdirectories you will need to specify wget that it should download your URL recursively. To do this you will need to add the -r option:

$ wget -r https://debaday.debian.net/

This command will generate a local mirror of the debaday blog. Note that wget respects the robots.txt file by default, if it exists. This means it will not download directories and files excluded by the robots.txt file.

Multiple URLs

Wget supports multiple URLs. You can either specify them in a file (one URL per line) or specify them in the command line (space separated).

$ wget url1 url2 ... urlN

or specify the URL container with the -i option

$ wget -i filePathAndName
Other options

Wget has a lot more options, you can for instance use:

  • -l for how deep the recursive download should go, the default depth is 5.
  • -c is invaluable as it allows you to continue an interrupted download
  • -O let’s you specify a target output file (-O fileName)

There are plenty other options in wget, the best way to know them is to read it’s rich man page. For those vista refugees amongst you try typing the following into a terminal ;)

$ man wget

Availability

As wget is part of the GNU project we assume it’s part of most Linux distribution. Never the less, official Debian and Ubuntu package are available:

  • Debian: stable, old stable, testing and unstable
  • Ubuntu: dapper, edgy, feisty and gutsy.

Community & developers

GNU wget is currently being maintained by Micah Cowan. The original author of GNU Wget is Hrvoje Nikšić.

Links

Posted in Debian, Ubuntu | 6 Comments »

Decibel Audio Player: An Audio Player for Human Beings

October 14th, 2007 edited by Vince

Article submitted by François Ingelrest. We are running out of articles! Please help DPOTD and submit good articles about software you like!

What is an Audio Player?

Aren’t you tired of those audio players with billions of useless features that clutter up their graphical interface? I am. Most of the time, the player looks good on paper, but when I’m faced with the interface, I don’t even know where to start in order to play my music. There are a lot of buttons, lists, combo boxes a bit everywhere and their usage is not so intuitive.

Above all, the worst thing about these features is that I really don’t need them. Do you really rate your tracks? I don’t. I know my music collection, I know what I like and what I don’t. Do you really need those fancy smart playlists? I don’t. When I want to listen to my music, I want to listen to an entire album, not just a track from here and another one from there.

A Solution Exists

Decibel Audio Player is a simple and nice audio player for the GNOME desktop. GNOME was precisely designed to be simple and nice, so why bother with a complex player and a cluttered graphical interface? Decibel Audio Player follows as closely as possible the GNOME Human Interface Guidelines, here is how it looks like:

Main Library

Another key aspect of this player is its modular design that allows users to enable only the features they want. The core of the player is very small and simple, and features are implemented in their own module that can be enabled/disabled as needed through the preferences:

Modules

So if you don’t want a feature, you simply disable the corresponding module and it gets out of your way. Decibel Audio Player is built on top of GStreamer, and currently supports MP3, Ogg Vorbis and FLAC formats. Its interface is intuitive, and tracks may be dragged, reordered, removed… as needed. Although it is still a young project, current features are correctly implemented and simply work as they should.

Availability

Decibel Audio Player is available in Debian unstable/testing and Ubuntu Gutsy Gibbon repositories (package decibel-audio-player). A more recent version, with music library support, is also available from the website. If you’re looking for a simple audio player, give it a try. You won’t be disappointed.

Links

Posted in Debian, Ubuntu | 9 Comments »

bc: an arbitrary precision numeric processing language

October 11th, 2007 edited by Alexey Beshenov

Article submitted by Markus Grunwald. We are running out of articles! Please help DPOTD and submit good articles about software you like!

Debian currently offers 84 packages that mention “calculator” (apt-cache search calculator | wc -l). An alternative to the more eye-candy-heavy calculators on offer is bc. bc has no fancy GUI, but is instead console-based and allows a simple mathematical language, making it really fast and programmable.

Everyday use

By default, bc behaves quite unhelpfully: zero digits and no predefined functions such as sin, cos, log etc. You can remedy this by calling bc with the -l parameter, so that it uses the standard math library. Instead of always typing “bc -l”, you can use the fact that bc reads the environment variable BC_ENV_ARGS. So for all the bash guys: add an export BC_ENV_ARGS=-l to your ~/.bashrc.

Let’s try the everyday use first: some simple calculations. Just type your expression and complete it with the enter key:

$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
850*77.1        <enter>
65535.0

That’s better than Excel 2007! And all of you terminal people will get this answer quicker than every KDE or GNOME user who starts their GUI calculator.

Custom scale and functions

The scale (total number of decimal digits after the decimal point) of bc’s results is 20 when you run it with “-l”, but you can set it up to a maximum of 2147483647. So to get a quite good estimate of π, just type:

scale=200  <enter>
4*a(1)     <enter>
3.141592653589793238462643383279502884197169399375105820974944592307\
81640628620899862803482534211706798214808651328230664709384460955058\
223172535940812848111745028410270193852110555964462294895493038196

Impressive. But what’s “a(1)”? bc uses very short names for the trigonometric functions. a(x) is the arctangent of x. For all of you who like longer names, you can define your own function and write it in a file. Add this file to your BC_ENV_ARGS and it will be read every time you start bc. The best thing to do is to get extensions.bc from http://x-bc.sourceforge.net/extensions_bc.html.

My BC_ENV_ARGS looks like this:

$ echo $BC_ENV_ARGS
-q -l /home/gru/.bc/extensions.bc

extensions.bc defines several functions by the names people normally know them, for example:

define sin(x)
{
        return (s(x))
}

This way, you can define your own function. I have to convert ratios to decibel and use this quite often:

define db(x,y)
{
        return 20*log(x/y);
}

Converting to other bases

bc can convert from and to arbitrary number bases. Say you want to convert from decimal to hexadecimal. Set the output base obase to 16:

obase=16 <enter>
10
A

And of course, you can calculate with these numbers too:

10/7
1.6DB6DB6DB6DB6DB6B

You can set the input base too:

ibase=16 <enter>
A/2
5
sin(AFFE) <enter>
-.B1F4021654E454E72

I leave the interpretation to the user ;)

Use in shell scripts

Bash is a nice shell, but when it comes to complex calculations $(()) won’t do the job. bc can help:

$ a=$( echo "l(1024)/l(2)" | bc )
$ echo $a
10

Conclusion

I searched quite long for the ideal calculator for me. After trying several GUI calculators, I didn’t find mine. Then I tried bc and it’s perfect for me. I hope you will like it too.

bc is available from Oldstable to Unstable in Debian and Ubuntu Dapper, Edgy, Feisty and Gutsy.

Links

Posted in Debian, Ubuntu | 4 Comments »

hpodder: a podcast client that just works.

October 7th, 2007 edited by Tincho

Article submitted by Matt Clare. We are running out of articles! Please help DPOTD and submit good articles about software you like!

Synopsis

hpodder is a command-line podcast client that just works. The command arguments are simple to master and allow for flexibility in downloading.

In depth

So you’ve have been desperate for a podcast client that rivals the ease and efficiency of iTunes (but have found Linux clients wanting) then hpodder may be the answer to your quest. Having found other clients to be a little jack-of-all-trades for your needs or else, just plain buggy, hpodder is a breath of fresh air.

Once installed, typing hpodder in the terminal will get things under way. The initial run will allow you to make a directory for access your podcasts from and give you the option to subscribe to the sample feeds. If you pass and just want to get onto subscribing your own feeds type:

$ hpodder add http://yourfavouritesite.com/podcastfeed.xml

This will add the feed to the hpodder database and assign an ID that allows for easy specification and manipulation of a feed later on. Hpodder will handle any valid podcast feed, even if it is not the traditional .xml address —including feeds extracted from an OPML export of iTunes podcasts. Once you have your list of podcasts you wish to download, simply type:

$ hpodder update
$ hpodder download

Or, if you want to make life easy:

$ hpodder fetch

As this does effectively the same job as the two commands combined. If you are using hpodder to listen to a podcast that you have perhaps been listening to on another machine, and want to download only the most recent episode, you can do this by typing:

$ hpodder catchup
$ hpodder fetch

If you want to only download or update specific podcasts, you can do so by specifying the podcast’s ID after the command (update, download, fetch, catchup). If you want to delete a podcast type:

$ hpodder rm ID

Where ID is the podcast’s ID number (else you risk deleting them all!).

Downloading the actual episode of your podcast is handled by cURL and is fast, efficient and if interrupted for whatever reason, can be picked up later on without having to start the download from the beginning.

If you want to make hpodder fetch on a regular basis, you can add it to your cron jobs and set it to run as frequently as you like.

If you want a full list of the commands and more subtle features available to hpodder users you can download the manual in PDF format, or read it online.

Any downsides?

The only real downside is that you need to use a second program to listen to your downloaded podcast episodes, but then most people have a favourite media player (kaffeine, mplayer, totem, vlc et al) that they use given the chance.

hpodder is available in Debian since Etch, and in Ubuntu since Feisty

Posted in Debian, Ubuntu | 4 Comments »

weather: check weather conditions and forecasts on the command line

October 4th, 2007 edited by Tincho

Article submitted by Dann Frazier. We are running out of articles ! Please help DPOTD and submit good articles about software you like !

weather, provided by the weather-util package, is a simple command line tool that lets you check current conditions and forecasts. It uses METAR data that it fetches from the the National Oceanic and Atmospheric Administration and forecasts from the National Weather Service. The documentation says it’s limited to providing information for the United States, but in fact, METAR data is available for stations all around the world. The forecast won’t be available, though.

Many locations are already pre-configured in the package’s /etc/weatherrc. If your location is already defined, you only need to look up the alias to begin using it. For example, /etc/weatherrc provides the following entry for Albuquerque, New Mexico:

[ABQ]
City = Albuquerque
ID = KABQ
St = NM

The alias for Albuquerque is ‘ABQ’. Therefore, weather ABQ will show you the current conditions for Albuquerque:

$ weather ABQ
Current conditions at Albuquerque International Airport (KABQ)
Last updated Sep 26, 2007 - 04:56 PM EDT / 2007.09.26 2056 UTC
  Wind: from the ESE (120 degrees) at 3 MPH (3 KT)
  Sky conditions: mostly clear
  Temperature: 77.0 F (25.0 C)
  Relative Humidity: 15%

weather -f ABQ will include a local forecast:

$ weather -f ABQ
Current conditions at Albuquerque International Airport (KABQ)
Last updated Sep 26, 2007 - 04:56 PM EDT / 2007.09.26 2056 UTC
  Wind: from the ESE (120 degrees) at 3 MPH (3 KT)
  Sky conditions: mostly clear
  Temperature: 77.0 F (25.0 C)
  Relative Humidity: 15%
City Forecast for Albuquerque, NM
Issued Wednesday afternoon - Sep 26, 2007
  Wednesday night... Low 52, 0% chance of precipitation.
  Thursday... Sunny, high 81, 0% chance of precipitation.
  Thursday night... Low 54, 5% chance of precipitation.
  Friday... Partly cloudy, high 80.
  Friday night... Low 56.

You can add new locations to this file directly, or to per-user ~/.weatherrc files. You can also define a default location by adding a stanza with an alias named “default”. For example, here’s the contents of my ~/.weatherrc file:

[default]
City = denver
Forecast = True
ID = KFNL
St = CO

City and St (state) are used for obtaining forecasts, while ID is used for retrieving current conditions. I use this configuration to tell weather to retrieve forecasts from Denver, CO but to use a station that is closer to me for obtaining current conditions. Note that if you do not specify a location and have no alias named ‘default’, weather will fall back to a hard-coded default of Raleigh, NC.

Knowing what information weather uses to obtain the data you request is important when defining your own configurations. For example, say I want to know the current conditions for Lubbock, TX and I run the following command:

$ weather -o --city=Lubbock --st=TX
Current conditions at Fort Collins-Loveland Municipal Airport (KFNL)
Last updated Sep 26, 2007 - 05:55 PM EDT / 2007.09.26 2155 UTC
  Wind: from the ENE (070 degrees) at 5 MPH (4 KT)
  Sky conditions: clear
  Temperature: 71 F (22 C)
  Relative Humidity: 23%

(The -o tells weather to omit the forecast). Notice that weather gave me the current conditions for my ‘default’ alias, not for Lubbock. This is because weather looks up current conditions by station ID, not by city/state. If I pass the station ID instead, I’ll get the information I was looking for:

$ weather -o --id=KLBB
Current conditions at Lubbock International Airport (KLBB)
Last updated Sep 26, 2007 - 05:53 PM EDT / 2007.09.26 2153 UTC
  Wind: from the S (170 degrees) at 9 MPH (8 KT) gusting to 18 MPH (16 KT)
  Sky conditions: mostly cloudy
  Weather: Cumulonimbus clouds, towering cumulus clouds observed
  Temperature: 86.0 F (30.0 C)
  Relative Humidity: 38%

Being a command-line tool, weather lends itself to various automatic processes. For example, you might want to cancel your weekly bicycle pub crawl if the forecast shows a high chance of rain:

$ cat /etc/cron.weekly/pub-reminder
#!/bin/sh

set -e

if weather -f | egrep -q '(100)|([5-9][0-9])% chance of precipitation’; then
 mailx -s ‘Cancellation’ $PUBCRAWLEES < ~/pub-crawl-cancelled
else
 mailx -s 'Get on your bike' $PUBCRAWLEES < ~/pub-crawl-is-a-go
fi

Of course, I’d just recommend a rain jacket.

weather is provided by the weather-util package and is available in Debian ‘etch’, testing and unstable, and Ubuntu ‘edgy’, ‘feisty’ and ‘gutsy’.

Edited to add: for international weather station ids, users can look on this US government website. Thanks hideokuze for the info.

Posted in Debian, Ubuntu | 14 Comments »

« Previous Entries Next Entries »