Convert MP3s to iPod Audio Book format (M4B)

I had the need to convert a group of mp3 files into a format that was suitable for playing on my iPod. Of course the mp3s could be played directly on the iPod without any trouble. This is great for songs, but an audio book is significantly longer. In my case I have a 40 minute commute each way and most audio books are too long to listen to during a commute. The iPod supports m4b files which are audio book files and they remember where they were stopped so you can resume listening to it after putting the iPod to sleep or listening to your music collection. The audio book format also supports changing the play back speed so it will be read to you much faster.

Mp3 based audio books usually come in mp3 chunks (about 10MiB or so). They can be converted into an audio book manually using the following steps:

  1. vbrfix (https://gna.org/projects/vbrfix) – Vbrfix reads the mp3 structure and rebuilds the file including a new Xing VBR header. This is applied to all the mp3s that comprise the audio book.
  2. mp3wrap (http://mp3wrap.sourceforge.net/) – Takes a list of mp3s and wraps them into one big one. The only thing to note is that the mp3s have to have a naming convention that allows them to be sorted properly at the command line. Otherwise mp3s could be placed in the wrong position.
  3. madplay streaming into faac (http://www.underbit.com/products/mad/ & http://www.audiocoding.com/) madplay is used to convert the output of mp3wrap into a wav file which is streamed into faac which creates the m4b file.
  4. aacgain (http://altosdesign.com/aacgain/) Takes the m4b file and applies a gain to it in an attempt to make it louder.

These steps can be performed manually, but it is tedious and error prone. I have written a python script that puts all of these together in an automated fashion.

  • The script takes a configuration file which:
    • Points to the directory containing the mp3 chunks
    • Points to a jpg or png file that represents the cover
    • Specifies an output name
    • Tag information
      • Artist
      • Year
      • Genre
      • Comment

A sample configuration file (typically named with the .cfg extension):

[mp3]
path=/mnt/media/iPod/unconverted/call_of_the_wild_64kb_mp3
coverart=/mnt/media/iPod/unconverted/call_of_the_wild_64kb_mp3/cover.jpg
outputfile=Jack London-Call of the Wild
artist=Jack London
title=Call of the Wild
year=1903
genre=AudioBook
comment=The Call of the Wild is a novel by American  writer Jack London. The plot concerns a previously domesticated  dog named Buck, whose primordial instincts return after a series of events leads to his serving as a sled dog in the Yukon during the 19th-century Klondike Gold Rush, in which sled dogs were bought at generous prices. Published in 1903, The Call of the Wild is London's most-read book, and it is generally considered his best, the masterpiece of his so-called "early period". Because the protagonist is a dog, it is sometimes classified as a juvenile novel, suitable for children, but it is dark in tone and contains numerous scenes of cruelty and violence. London followed the book in 1906 with White Fang, a companion novel with many similar plot elements and themes as Call of the Wild, although following a mirror image plot in which a wild wolf becomes civilized by a mining expert from San Francisco named Weedon Scott.The Yeehat, a group of Alaska Natives portrayed in the novel, are a fiction of London's.

Note: Wikipedia is an excellent source of biographical material

Typically, a number of configuration files are created so audio books can be created unattended in a batch.

The script features:

  • logging capabilities – successes and failures are logged. If a failure occurs in a conversion during a batch operation it is easy to track it down
  • Checks to see if all required components are available to the script. If not it prompts for the required components. It even provides an apt-get string for Ubuntu that can be used to install the required components
  • Fixes an vbr inconsistencies
  • wraps the mp3s into one large mp3 – beware that the mp3s need to be properly named i.e. they need to be named so that when they are sorted by the operating system they are in the correct order
  • Tags the resulting m4b file with artist, comment, genre, year and cover art. Tagging the cover art is particularly nice as it shows up in the iPod

mp3tom4b.py:

#!/usr/bin/env python
#-*- coding:utf-8 -*-

"""
This script will take a folder and attempt to convert the mp3s within it to m4b
files (iPod audiobook format).

1) The mp3s are processed using vbrfix
2) The mp3s are joined using the mp3wrap
2) It will encode the newly joined mp3 to m4b
3) The wrapped mp3 will be removed

The output file will be placed in a sub folder of the mp3 folder.

Note: all of the mp3s to be joined as part of the conversion must be in the same
folder and they must have a number or identifier that allows them to be sorted
properly i.e. a proper string sort.

Documentation:

References:

Dependencies:
    vbrfix - https://gna.org/projects/vbrfix
    mp3wrap - http://mp3wrap.sourceforge.net/
    madplay - http://www.underbit.com/products/mad/ - This is a decoder used to
    convert the mp3 to wave
    faac - http://www.audiocoding.com/ - convert wav file to m4b format
    aacgain - http://altosdesign.com/aacgain/

TODO:

License:
The MIT License

Copyright (c) 2010 Troy Williams

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
"""

import sys
import os
import subprocess
import ConfigParser
import logging

#Constants
__uuid__ = '62a5aa15-2f1f-40e8-8a01-2a5cc74f6fb6'

__version__ = '0.6'
__author__ = 'Troy Williams'
__email__ = 'troy.williams@bluebill.net'
__copyright__ = 'Copyright (c) 2010, Troy Williams'
__date__ = '2010-04-05'
__license__ = 'MIT'
__maintainer__ = 'Troy Williams'
__status__ = 'Development'

#script Level Variables
mainLogger = None

def initialize_log_options():
    """
    Creates a dictionary with the proper values to pass to the logging object

    Dictionary keys:
    level - the debug level to display in the log file
    name - the name of the logger
    quiet - whether to display log messages to the screen - Default=False
    clean - deletes the log file if it exists - Default=True
    log file - the log file to use
    """

    options = {'level' : 'info',
               'name' : 'Log Tester',
               'quiet' : False,
               'clean' : False,
               'log file' : None}
    return options

def initialize_logging(options):
    """
    Log information based upon users options

    options is a dictionary that contains the various log options - see
    initialize_log_options for details

    StackOverflow.com Attribution:
    http://stackoverflow.com/questions/616645/how-do-i-duplicate-sys-stdout-to-a-log-file-in-python/648322#648322
        User Profile: http://stackoverflow.com/users/48658/atlas1j

    Note: The initialize_logging function is only used and it has been modified
          to use a dictionary instead of optparse options class.

    Levels:
    Logger.debug()
    Logger.info()
    Logger.warning()
    Logger.error()
    Logger.exception() <- same as error except provides a stack trace
    Logger.critical()
    """

    if not options:
        raise Exception, 'No logging options set...'

    logger = logging.getLogger(options['name'])
    formatter = logging.Formatter('%(asctime)s %(levelname)s\t%(message)s')
    level = logging.__dict__.get(options['level'].upper(), logging.DEBUG)
    logger.setLevel(level)

    # Output logging information to screen
    if not options['quiet']:
        hdlr = logging.StreamHandler(sys.stderr)
        hdlr.setFormatter(formatter)
        logger.addHandler(hdlr)

    # Output logging information to file
    logfile = options['log file']
    if options['clean'] and os.path.isfile(logfile):
        os.remove(logfile)
    hdlr2 = logging.FileHandler(logfile)
    hdlr2.setFormatter(formatter)
    logger.addHandler(hdlr2)

    return logger

def which(program):
    """
    Takes a binary file name as an argument and searches the path(s) for it. If
    found, the full path is returned. Else None is returned

    StackOverflow.com Attribution::
    http://stackoverflow.com/questions/377017/test-if-executable-exists-in-python/377028#377028
        User Profile: http://stackoverflow.com/users/20840/jay
    """

    def is_exe(fpath):
        return os.path.exists(fpath) and os.access(fpath, os.X_OK)

    fpath, fname = os.path.split(program)
    if fpath:
        if is_exe(program):
            return program
    else:
        for path in os.environ['PATH'].split(os.pathsep):
            exe_file = os.path.join(path, program)
            if is_exe(exe_file):
                return exe_file
    return None

def BuildAptGet(programs):
    """
    Takes the list of programs, a tupple of two values - program name and url,
    and builds an apt get string.

    returns a sudo apt-get string that a user could use to install the required
    components on Ubuntu Linux
    """
    install = []
    if programs:
        for p in programs:
            install.append(p[0])

        return 'sudo apt-get install ', ' '.join(install)

def CheckDependencies():
    """
    Checks the current operation system to see if the dependencies are available
    and installed. An error is raised if the program doesn't exist
    """

    programs = []
    #mp3wrap - http://mp3wrap.sourceforge.net/
    programs.append(('mp3wrap', 'http://mp3wrap.sourceforge.net/'))

    #faac - http://www.audiocoding.com/ - convert wav file to m4b format
    programs.append(('faac', 'http://www.audiocoding.com/'))

    #madplay - http://www.underbit.com/products/mad/ - This is a decoder used to
    #convert the mp3 to wave
    programs.append(('madplay','http://www.underbit.com/products/mad/'))

    #vbrfix - https://gna.org/projects/vbrfix
    programs.append(('vbrfix','http://gna.org/projects/vbrfix'))

    #aacgain - http://altosdesign.com/aacgain/
    programs.append(('aacgain','http://altosdesign.com/aacgain/'))

    #loop through the programs and see if they exist. If they do not, then
    #add them to the missing list
    missing = []
    for p in programs:
        if not which(p[0]):
            missing.append(p)

    #If there are any missing programs, create a printable list
    #and raise an exception
    if missing:
        messages = []
        for p in missing:
            messages.append('%s not found! Please install see %s for details'
                            % p)
        print 'Missing files:'
        print messages
        #Build the aptget string suitable for Ubuntu
        aptGet = BuildAptGet(missing)
        print 'If using Ubuntu you can execute this line to install missing programs:'
        print aptGet

        raise Exception, 'Missing critical programs...'

def makeDirectory(dir_path):
    """
    Takes the passed directory path and attempts to create it including all
    directories or sub-directories that do not exist on the path.
    """

    try:
        os.makedirs(dir_path)
    except OSError:
        #Check to see if the directory already exists
        if os.path.exists(dir_path):
            #It exists so ignore the exception
            pass
        else:
            #There was some other error
            raise

def process_command_line():
    """
    From the Docs: http://docs.python.org/library/optparse.html
    """
    from optparse import OptionParser

    usage = """
            usage: %prog [options] file

            This script will take a series of mp3 files and combine them to form
            an iPod audio book (.m4b) file. It will join the mp3's using
            mp3wrap. It will then run vbrfix to correct any issues. After that
            mp3gain will be used to increase the volume of the mp3 file. Finally
            faac will  be used to convert the mp3 to m4b and tag it with the
            appropriate information.

            file - the name of the configuration file that holds the information
            about the mp3's to be converted to an audiobook. It should look
            somthing like this:
            #-------------------------------
            [mp3]
            path=/path/to/mp3s
            coverart=/path/to/mp3s/cover.jpg
            outputfile=output-audiobook
            artist=Author
            title=book title
            year=2010
            genre=AudioBook
            comment=Some comments about the book
            #-------------------------------

            where:
            path - the absolute path to the mp3s that comprise the audio book
            outputfile - the name of the final output file
            artist - the author of the book
            title - the title of the book
            year - the year the book was published
            genre - should be set to AudioBook or some appropriate genre
            coverart - the absolute path to the image used as the book cover
            """
    parser = OptionParser(usage=usage, version='%prog v' + __version__)

    options, args = parser.parse_args(args=None, values=None)

    if len(args) != 1:
        parser.error('Only one configuration file is required')
        parser.print_help()

    return options, args

def RunCommand(command, useshell=False):
    """
    Takes the list and attempts to run it in the command shell.

    Note: all bits of the command and paramter must be a separate entry in the
    list.
    """
    if not command:
        raise Exception, 'Valid command required - fill the list please!'

    p = subprocess.Popen(command, shell=useshell)
    retval = p.wait()
    return retval

def loadConfigParameters(path):
    """
    Takes a path to a configuration file and reads in the values stored there.

    Returns: dictionary
    """

    if not os.path.exists(path):
        raise Exception, '%s does not exist!' % path

    #Set the defaults
    configParams = {}
    configParams['path'] = None
    configParams['outputfile'] = None
    configParams['artist'] = None
    configParams['title'] = None
    configParams['album'] = None
    configParams['year'] = None
    configParams['comment'] = None
    configParams['genre'] = None
    configParams['track'] = None
    configParams['coverart'] = None

    config = ConfigParser.RawConfigParser()
    config.read(path)

    #loop through all the items in the section and assign the values to the the
    #configParams dictionary... We don't assign it as the default dictionary
    #because, the options we are interested in are defined above... This
    #appears to be case sensitive therefore we make the keys lower case
    for name, value in config.items('mp3'):
        configParams[name.lower()] = value

    return configParams

def find_mp3s(path):
    """
    Takes the folder and returns a list of mp3s in that folder.

    Returns a sorted list of files with the full path name.
    """
    files = []
    for i in os.listdir(path):
        filename = os.path.join(path, i)
        if os.path.isfile(filename):
            basename, ext = os.path.splitext(filename)
            if ext.lower() == '.mp3':
                files.append(filename)

    files.sort()
    return files

def fixMP3Bitrate(mp3Path, outputdirName):
    """
    mp3Path - the path to the directory contain the mp3s that will be adjusted
    by vbrFix

    outputdirName - the name of the directory to store the fixed mp3s - will be
    a subdirectory
    """
    if not os.path.exists(mp3Path):
        raise Exception, '%s does not exist!' % mp3Path

    outputPath = os.path.join(mp3Path, outputdirName)

    #make the output directory
    makeDirectory(outputPath)

    #fix the bit rate on each and every mp3 that comprises the audio book -
    #copying the modified files to the output directory
    mp3files = find_mp3s(mp3Path)

    if not mp3files:
        raise Exception, '%s does not contain mp3s!' % mp3Path

    command = []
    for mp3 in mp3files:
        (dirName, fileName) = os.path.split(mp3)
        newpath = os.path.join(outputPath, fileName)
        command = ['vbrfix', '-allways']
        command.append('%s' % mp3)
        command.append('%s' % newpath)
        RunCommand(command)

def pathExists(path):
    """
    takes a tupple that contains a folder path and file name and attempts
    to determine if it exists
    """

    filepath, filename = path
    fullpath = os.path.join(filepath, filename)

    return os.path.exists(fullpath)

def wrapMP3(path):
    """
    Takes the path to a directory containing mp3s to wrap into one mp3

    returns a tupple containing the path and filename of the wrapped mp3
    """

    if not os.path.exists(path):
        raise Exception('Path does not exist!')

    filename = 'wrap'
    output = os.path.join(path, '%s.mp3' % filename)

    command = ['mp3wrap', '-v', '%s' % output]

    files = find_mp3s(path)

    if files:
        #append the files to the command list
        command = command + files
    else:
        raise Exception, 'No mp3 files to wrap!'

    RunCommand(command)

    return (path,'%s_MP3WRAP.mp3' % filename)

def adjust_aac_gain(path):
    """
    Takes a tupple of file path and file name to an aac to adjust the gain
    using aacgain
    """

    filepath, filename = path
    fullpath = os.path.join(filepath, filename)

    if not os.path.exists(fullpath):
        raise Exception, 'Path does not exist!'

    command = ['aacgain']
    command.append('-r')
    command.append('-k')
    command.append('%s' % fullpath)

    RunCommand(command)

    return path

def convert_m4b(path, configParams = None):
    """
    Takes a tupple representing a file path and file name of an mp3
    and attempts to convert it to an m4b file.

    It returns a tupple containing the file path and filename of the results
    """

    filepath, filename = path
    fullpath = os.path.join(filepath, filename)
    mainLogger.debug('Path to mp3 to convert to m4b = %s' % fullpath)

    if not os.path.exists(fullpath):
        raise Exception, 'Path does not exist!'

    output = 'converted.m4b'

    commandMadPlay = ['nice', '-10']
    commandMadPlay.append('madplay')
    commandMadPlay.append('-q')
    commandMadPlay.append('-o')
    commandMadPlay.append('wave:-')
    commandMadPlay.append('%s' % fullpath)

    commandfaac = ['nice', '-10']
    commandfaac.append('faac')
    commandfaac.append('-w')

    if configParams:
        if configParams['artist']:
            commandfaac.append('--artist')
            commandfaac.append('%s' % configParams['artist'])

        if configParams['title']:
            commandfaac.append('--title')
            commandfaac.append('%s' % configParams['title'])

        if configParams['album']:
            commandfaac.append('--album')
            commandfaac.append('%s' % configParams['album'])

        if configParams['year']:
            commandfaac.append('--year')
            commandfaac.append('%s' % configParams['year'])

        if configParams['comment']:
            commandfaac.append('--comment')
            commandfaac.append('%s' % configParams['comment'])

        if configParams['genre']:
            commandfaac.append('--genre')
            commandfaac.append('%s' % configParams['genre'])

        if configParams['track']:
            commandfaac.append('--track')
            commandfaac.append('%s' % configParams['track'])

        if configParams['coverart']:
            commandfaac.append('--cover-art')
            commandfaac.append('%s' % configParams['coverart'])

    commandfaac.append('-q')
    commandfaac.append('80')
    commandfaac.append('-o')
    commandfaac.append('%s' % os.path.join(filepath, output))
    commandfaac.append('-')

    mainLogger.debug('madplay cmd line = %s' % subprocess.list2cmdline(commandMadPlay))
    mainLogger.debug('faac cmd line = %s' % subprocess.list2cmdline(commandfaac))

    madplayProcess = subprocess.Popen(commandMadPlay, shell=False,
                                                      stdout=subprocess.PIPE)
    faacProcess = subprocess.Popen(commandfaac, shell=False,
                            stdin=madplayProcess.stdout, stdout=subprocess.PIPE)
    retval = faacProcess.wait()

    return (filepath, output)

def main():
    """
    Take a number of mp3 bits that comprise an audiobook and convert it to
    an m4b file - an iPod audiobook file format
    """

    global mainLogger #make sure that other methods can use the log

    logoptions = initialize_log_options()
    #NOTE: the options can be pulled from the command line arguments
    logoptions['log file'] = os.path.join(sys.path[0], sys.argv[0] + '.log')
    #options['clean]' = True

    # Setup logger format and output locations
    mainLogger = initialize_logging(logoptions)

    #grab the command line arguments
    options, args = process_command_line()
    mainLogger.debug('len(args) = %s' % len(args))

    mainLogger.info('Loading Configuration Parameters...')
    configParams = loadConfigParameters(args[0])

    #The working folder under the mp3 path
    outputdir = 'output'

    try:
        mainLogger.info('Checking Dependencies...')
        CheckDependencies()

        mainLogger.info('Working on %s' % configParams['path'])
        mainLogger.info('Validating Configuration Parameters...')
        if not os.path.exists(configParams['path']):
            raise Exception, '%s does not exist!' % configParams['path']

        mainLogger.info('Fixing mp3 bitrate...')
        fixMP3Bitrate(configParams['path'], outputdir)

        path = os.path.join(configParams['path'], outputdir)
        mainLogger.debug('Output folder = %s' % path)
        mainLogger.info('Combining mp3s into one big one...')
        output = wrapMP3(path)

        if not pathExists(output):
            raise Exception, 'The wrapped mp3 does not exist!'

        #convert the mp3 to m4b
        mainLogger.info('Converting to audiobook...')
        output = convert_m4b(output, configParams)
        mainLogger.debug('m4b = %s/%s' % output)

        if not pathExists(output):
            raise Exception, 'conversion result does not exist!'

        #rename the output file
        source =  os.path.join(output[0], output[1])
        dest = os.path.join(output[0], '%s.m4b' % configParams['outputfile'])

        mainLogger.info('Renaming the audio book...')
        mainLogger.debug('rename %s to %s' % (source, dest))

        os.rename(source, dest)
        output = (output[0], '%s.m4b' % configParams['outputfile'])

        #adjust the gain of the audiobook
        mainLogger.info('Adjusting the gain...')
        output = adjust_aac_gain(output)

        mainLogger.info('completed %s/%s' % output)
    except Exception as inst:
        mainLogger.error(inst, ' Occured while processing ', configParams['path'])
        mainLogger.exception(inst, configParams)
        return 1

    finally:
        #Clean up the files by deleting everything in the output folder except
        #for the .m4b file
        searchFolder = os.path.join(configParams['path'], outputdir)
        files = []
        if os.path.exists(searchFolder):
            for i in os.listdir(searchFolder):
                f = os.path.join(searchFolder, i)
                if os.path.isfile(f):
                    ext = os.path.splitext(f)[1]
                    if ext.lower() != '.m4b':
                        files.append(f)
            [os.remove(f) for f in files]

    return 0

if __name__ == '__main__':
    status = main()
    sys.exit(status)

Here is an example of a shell script that can be created to call the conversion script:

#!/bin/sh
#A simple shell script to call the mp3 to m4b conversion script on various cfg files
./convertMP3toM4b.py cfgs/callofthewild.cfg
Advertisements

3 thoughts on “Convert MP3s to iPod Audio Book format (M4B)

  1. Ash says:

    Thanks. This is what I was looking for but I am a newbie and not a technical person. Is there some way to have all the steps you listed through some easy steps which a layman like me can understand? Thanks

    • The script was designed with Ubuntu Linux in mind and has not been tested on any other operating system. It should work on other Linux systems without too much trouble.

      You’ll need to copy the script to a file and give it execute permissions.

      Next you need to create a .cfg file that contains the information that the script requires about the audio book.

      You’ll need to execute the script (see the shell script for an example of how to format the command line) and pass it the path to the configuration file.

      Typically I create 4 or 5 cfg files and then add them to the shell script for batch execution.

      Cheers,
      Troy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s