Python Training – Part 4

Part 1 | Part 2 | Part 3 || Part 5

This is part 4 of a Python training that I gave while I was working at SPIELO International. My manager kindly gave me permission to publish the material. The material is Copyright © 2008-2011 SPIELO International, licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Advertisement

More Common Scripting Tasks

Parsing Command-Line Arguments

The command-line arguments to the Python program can be found in the “sys.argv” variable.

When you start a Python program with this command line:

C:> tempcmd_line.py -x -o “the output.txt” input.txt

The contents of “sys.argv” are as follows:

[‘C:\temp\cmd_line.py’, ‘-x’, ‘-o’, ‘the output.txt’, ‘input.txt’]

The “optparse” module provides more convenient access to the command-line arguments:

  • You can specify a list of supported arguments along with their data types
  • You can specify default values for omitted arguments
  • The usage screen (“–help” or “-h”) is generated automatically
  • …and much more

As an example, let’s use “optparse” to interpret the arguments of this GNU program:

> head -h
Usage: head [OPTION]... [FILE]...
Print first 10 lines of each FILE to standard output.
With more than one FILE, precede each with a header giving the file name.
With no FILE, or when FILE is -, read standard input.

  -c, --bytes=SIZE         print first SIZE bytes
  -n, --lines=NUMBER       print first NUMBER lines instead of first 10
  -q, --quiet, --silent    never print headers giving file names
  -v, --verbose            always print headers giving file names
      --help               display this help and exit
      --version            output version information and exit

...

Report bugs to <bug-textutils@gnu.org>.

What we see:

  • The program accepts a list of options (using a “-“ or “–” prefix) and arguments (the list of files)
  • There are options with and without parameters (“-n” takes the number of lines, while “-v” does not require a parameter)
  • Some options have default values.

Here’s a Python program with a command-line interface like that:

import optparse
import sys

def Main():
    parser = optparse.OptionParser(
        usage="Usage: %prog [OPTION]... [FILE]...",
        version="Version 1.0.nWritten by me.",
        description="Print first 10 lines of each FILE ...",
        epilog="... Report bugs to <bug-textutils@gnu.org>.")
    parser.add_option("-c", "--bytes", type="int",
                      metavar="SIZE",    # Here, the default
                                         # metavar would be "BYTES"
                      help="print first SIZE bytes")
    parser.add_option("-n", "--lines", type="int",
                      metavar="NUMBER", dest="num_lines",
                      help="print first NUMBER lines instead of first 10")
    parser.add_option("-q", "--quiet", "--silent",
                      action="store_true",
                      help="never print headers giving file names")
    parser.add_option("-v", "--verbose", action="store_true",
                      help="always print headers giving file names")

    parser.set_defaults(num_lines=10, quiet=False, verbose=False)
    options, args = parser.parse_args()

    print "-c =", options.bytes
    print "-n =", options.num_lines
    print "-q =", options.quiet
    print "-v =", options.verbose
    print "args =", args

if __name__ == "__main__":
    sys.exit(Main())

How it works:

  • To define the command-line interface, create an “optparse.OptionParser” instance and invoke its methods.
  • The “OptionParser” initializer takes a number of optional keyword arguments for things like version info and help text.
  • You add options using the “add_option” method:
    • The first arguments specify the short and long option strings.
    • Additional keyword arguments specify things like data type and help string.
  • Default values are best set using the “set_defaults” method.
  • The “parse_args” method returns an object that contains the option values as attributes and a list of positional arguments.

Try to invoke the program using the following command lines:

> cmd_line.py -h

> cmd_line.py somefile

> cmd_line.py –n 12 somefile

> cmd_line.py –n this_is_not_a_string

> cmd_line.py –unknown-option

Reading from INI Files

The “ConfigParser” module can be used to read and write INI files.

Here’s an example INI file:

; Example INI file
[Basic]
quiet=True
lines=10
multiline=This is
 a multi-line value

[Files]
dir=c:temp
# %(dir)s will be replaced with the value of dir
input=%(dir)sinput.txt

This INI file can be read as follows:

import ConfigParser

if __name__ == "__main__":
    p = ConfigParser.SafeConfigParser()
    p.read(["config.ini"])
        # read() can load several INI files at once.
    print "multiline =", p.get("Basic", "multiline")
    print "quiet =", p.getboolean("Basic", "quiet")
    print "lines =", p.getint("Basic", "lines")
    print "input =", p.get("Files", "dir")
    print "input =", p.get("Files", "input")
    try:
        p.get("Basic", "non-existant")
    except (ConfigParser.NoSectionError,
            ConfigParser.NoOptionError), e:
        print e
    print "Sections:", p.sections()
    print "Items in Basic:", p.items("Basic")

Creating and Reading ZIP Files

Use the “zipfile” module to work with ZIP files.

Create a new archive:

The following code creates a new archive with two files:

  • One file is read from a file on disk and stored under a different name in the archive.
  • The other file is constructed directly from a Python string (which could also contain binary data).
import zipfile

z = zipfile.ZipFile("new_archive.zip", "w",
                    compression=zipfile.ZIP_DEFLATED)
z.write("file.txt",             # This file on disk...
        "subdir/t.txt")         # ...is added under this name.
z.writestr("text.txt",          # Name in the archive
           "Specify the contents of the file directly")
z.close()

Add files to an existing archive:

To add files to an existing archive, specify mode “a” when opening the file:

z = zipfile.ZipFile("existing_archive.zip", "a")
z.write(...

Read an existing archive:

Using the “infolist” method, you can retrieve a list of “ZipInfo” objects for each file in the archive. Using the “read” method, you can retrieve the byte stream of a file as a Python string:

z = zipfile.ZipFile("new_archive.zip", "r")
for i in z.infolist():
    print i.filename, i.compress_size, i.file_size, "etc."
    print "File contents:", z.read(i.filename)
z.close()

Note: See also the modules “tarfile”, “gzip”, “bz2”, and “zlib” for other ways of creating archives and compressing data.

Interfacing with C++

There are many levels on which you can use Python and C++ (or other programming languages) together:

  • Interpret binary data (potentially produced by C++)
  • Invoke functions in a C++ DLL from Python
  • Write Python extension modules in C++ (advanced)
  • Embed the Python interpreter in C++ to offer scripting facilities (advanced)

Working with Binary Data

Let’s assume you have a C++ program that writes the following struct to a binary file:

struct TestStructure
{
    unsigned char ByteMember;
    signed short ShortMember;
    char StringBuffer[11];
    unsigned long LongMember;
};

void PrintTestStructToFile(const char* filename)
{
    TestStructure t;
    t.ByteMember = 1;
    t.ShortMember = -1;
    strcpy(t.StringBuffer, "abcdefg");
    t.LongMember = 0xcafebabe;

    FILE* f = fopen(filename, "w");
    fwrite(&t, sizeof(t), 1, f);
    fclose(f);
}

When you open the file in binary mode, you might get a string like this:

‘x01xccxffxffabcdefgx00xccxccxccxccxbexbaxfexca’

We want something else.

First, use the “ctypes” module and re-define the struct in Python:

import ctypes
class TestStructure(ctypes.Structure):
    _fields_ = [("ByteMember", ctypes.c_ubyte),
                ("ShortMember", ctypes.c_short),
                ("StringBuffer", ctypes.c_char * 11),
                ("LongMember", ctypes.c_ulong)]

Next, we can read the file into a “ctypes” byte buffer and “cast” it to the struct type:

data = ctypes.create_string_buffer(
    open(filename, "rb").read())

struct = TestStructure.from_address(ctypes.addressof(data))
    # Of course, we can also create an uninitialized instance
    # by writing "struct = TestStructure()".
print "ByteMember", struct.ByteMember
print "ShortMember", struct.ShortMember
print "StringBuffer", struct.StringBuffer
print "LongMember", hex(struct.LongMember)

# When initializing the struct from a pointer to a data buffer,
# the buffer must live at least as long as we use the struct.
# Therefore, store a reference to "data" right in "struct".
struct.data = data

We can also save the data back to a binary file from Python:

open(filename, "wb").write(
    ctypes.string_at(ctypes.addressof(struct),
                     ctypes.sizeof(struct)))

See the help for the “ctypes” module for more information.

Note: You can also use the “struct” module for working with binary data.

Invoking Functions in a C++ DLL

Let’s assume we have a DLL that exports the following function:

extern "C" __declspec(dllexport)
const char* __stdcall WorkWithFile(
    const char* filename)
{
    printf("Doing something with %sn", filename);
    return "It worked!";
}

We can invoke it from Python like this:

dll = ctypes.WinDLL("cpp_code.dll")
dll.WorkWithFile.restype = ctypes.c_char_p
print dll.WorkWithFile("some_file.txt")

Things to note:

  • We’re using “ctypes.WinDLL”, because we want to call a “__stdcall” function. (We’d use “ctypes.CDLL” for “__cdecl” functions.)
  • By default, “ctypes” assumes that the return type of the function is an integer. By setting the “restype” attribute of the function wrapper, we can specify the real return type.

What if you want to use C++ classes exported from a DLL?

You should compile the C++ code as a Python extension module. See the next section.

(Exporting classes directly from a DLL is generally not a good idea. This approach is not portable across different compilers, or even different versions of the same compiler. Everything you export should be declared as “extern “C”” to avoid problems with name mangling.)

Extending and Embedding Python

Many of the modules in the standard Python library are actually extension modules written in C.

Wrapping up some C++ code as an extension module is a task that can be largely automated using the SWIG program. See my article “Python Extensions in C++ Using SWIG”.

It is also possible to embed the Python interpreter in a C++ program. This is especially useful if you want to provide a simple way for users to write plug-ins for your program, or to provide a built-in scripting language (like VBA for Microsoft Office).

In the mathematics department, we’ve been using the PyCXX C++ library successfully for this task. See http://cxx.sourceforge.net/.

Introduction to GUI Programming

wxPython Demo

There are many GUI toolkits available for Python. In the mathematics department, we’re using wxPython exclusively (http://www.wxpython.org/), which is a binding for the cross-platform wxWidgets library (http://www.wxwidgets.org/). wxPython allows us to create complex, state-of-the-art GUIs relatively easily (HOMER being the most recent example).

Once you have installed wxPython to your local Python installation, you should take a look at the wxPython Demo (usually in Start → Programs →wxPython2.8 Docs Demos and Tools → Run the wxPython DEMO).

In the next few sections, we’ll use wxPython to build some very simple GUIs to make your scripts easier to use for people who don’t know what the command line is. 😉

Dialog Boxes in Command-Line Programs

Sometimes you have a command-line program and just want to ask the user for a filename, pick an item from a list, enter a string, or whatever. This can be done easily.

Add this code to your program:

import wx
import wx.lib.dialogs as dlg
g_app = wx.PySimpleApp()    # This object must be alive as long as
    # you want to open dialogs. Without an app object, the program
    # will crash.

Usability note: Consider adding a command-line or INI file-based interface to your program in addition to the GUI. This way, the program can be run unattended from a script, without having to make the same choices manually each time the program is run.

Displaying Messages

To display a simple message box with an OK button, use this code:

dlg.messageDialog(message="Message", title="Title", aStyle=wx.OK)

You can also display other buttons or add an icon:

result = dlg.messageDialog(
    message="Are you sure?", title="The Tool",
    aStyle=wx.YES | wx.NO | wx.ICON_WARNING)
if result.returned == wx.ID_YES:
    print "As you wish, Master!"

To display a longer text and/or to allow the user to copy the text to the Clipboard, use this:

dlg.scrolledMessageDialog(
    message="This is some long textn" * 100, title="The Tool")

Asking for Some Text

Example usage:

result = dlg.textEntryDialog(title="Enter something",
                             message="Right here",
                             defaultText="default")
if result.accepted:
    print "You entered", result.text
else:
    print "Cancelled"

Asking for Files and Directories

Asking for files to open:

result = dlg.openFileDialog(title='Open',
    directory='c:\temp', filename='x.txt',
    wildcard='Text Files (*.txt)|*.txt',
    style=wx.OPEN | wx.MULTIPLE)
if result.accepted:
    print "You selected", result.paths

Asking for a file to save:

result = dlg.saveFileDialog(title='Save',
    directory='c:\temp', filename='x.txt',
    wildcard='Text Files (*.txt)|*.txt',
    style=wx.SAVE | wx.OVERWRITE_PROMPT)
if result.accepted:
    print "You selected", result.paths

Asking for a directory:

result = dlg.dirDialog(message='Choose a directory',
                       path='c:\temp')
if result.accepted:
    print "You selected", result.path

Offering Multiple Choices

To allow the user to pick a single item:

result = dlg.singleChoiceDialog(message='Choose wisely',
                                title='The Tool',
                                lst=['Blue pill', 'Red pill'])
if result.accepted:
    print "Your choice:", result.selection

To allow the user to pick multiple items:

result = dlg.multipleChoiceDialog(message='Choose',
    title='The Tool', lst=['Cheese', 'Ham', 'Mushrooms'])
if result.accepted:
    print "Your choice:", result.selectio

A Minimal GUI Application

Here’s the code to open a main window:

import wx

class MainFrame(wx.Frame):
    pass

class App(wx.App):
    def OnInit(self):
        frame = MainFrame(parent=None,
                          title="The GUI")
        frame.Show()
        return True

if __name__ == "__main__":
    app = App(redirect=False)
    app.MainLoop()

What we see:

  • To create a GUI application, derive a class from “wx.App”, instantiate it, and call its “MainLoop” method.
  • In the “OnInit” method, the App object creates the main frame.
  • We create the App object with “redirect=False”. This means that all messages (including the output of “print” statements) will be printed to the console. This is useful during debugging when you start the program from the console. When you set “redirect=True”, wxPython creates a separate log window to display messages.

Adding Widget Inspector and an Interactive Shell

During development, you’d often wish to peek into the program while it’s running, just like you could in an interactive Python shell. This can be done easily with the “InspectionTool” that wxPython provides.

Let’s add the code to display a button and to open the “InspectionTool” when you click the button:

from wx.lib.inspection import InspectionTool

class MainFrame(wx.Frame):
    def __init__(self, parent, title):
        wx.Frame.__init__(self, parent=parent, title=title)

        inspector_btn = wx.Button(self, -1, "Widget Inspector")
        self.Bind(wx.EVT_BUTTON, self.OnOpenWidgetInspector,
                  inspector_btn)

        self.m_hello = "World"

    def OnOpenWidgetInspector(self, evt):
        if not InspectionTool().initialized:
            InspectionTool().Init()
        InspectionTool().Show(self, True)
wxPython Widget Inspector

What we see:

  • To create a button, create a “wx.Button” object. The first parameter to the initializer is the parent window. The second one is a unique ID (used to distinguish widgets in event handlers). We don’t need an ID, so we just set it to -1. The third parameter is the button label. For the other optional parameters, please see the wxPython Docs.
  • To register an event handler that should be called when the button is pressed, use the “Bind” method with the ID of the event and the method to invoke.
  • The event handler method takes a “wxEvent” objects as its only parameter. We don’t need the event object in our case.

When you run the application and click the button, the Widget Inspector opens. You can browse the GUI widgets that you created and use the interactive shell to work with the objects.

Working with Sizers

Example GUI

Sizers are used in wxPython to calculate the layout of widgets. Sizers perform the following tasks automatically:

  • Arrange widgets horizontally, vertically, in a grid, etc.
  • Resize widgets when the parent window is resized
  • Adjust the size of the parent to the space requirements of the children

As an example for working with sizers, let’s build a GUI with two buttons and a text box. The buttons should be right-aligned and have a nice border around them. The text box should consume the remaining free space. When the frame is resized, the layout should adapt.

The screenshot to the right shows the desired result.

For this layout, we use two sizers, a vertical one with two compartments and a horizontal one with three:

GUI Sizers

First, we create the widgets normally:

class MainFrame(wx.Frame):
    def __init__(self, parent, title):
        wx.Frame.__init__(self, parent=parent, title=title)
        self.SetBackgroundColour(
            wx.SystemSettings_GetColour(wx.SYS_COLOUR_BTNFACE))

        inspector_btn = wx.Button(self, -1, "Widget Inspector")
        self.Bind(wx.EVT_BUTTON, self.OnOpenWidgetInspector,
                  inspector_btn)

        quit_btn = wx.Button(self, -1, "Quit")

        text_box = wx.TextCtrl(self, -1, style=wx.TE_MULTILINE,
                               size=(500, 300))

Next, we create a horizontal sizer for the buttons:

horz_sizer = wx.BoxSizer(wx.HORIZONTAL)
        horz_sizer.AddStretchSpacer(prop=1)
        horz_sizer.Add(inspector_btn, proportion=0,
                       flag=wx.RIGHT, border=4)
        horz_sizer.Add(quit_btn, proportion=0)

What we see:

  • To right-align the buttons, we add a “stretch spacer” first. The argument “prop=1” defines the proportion. This will be explained shorty.
  • Next, we add the “Widget Inspector” button and add 4 pixels of free space to its right. The argument “proportion=0” will be explained shortly.
  • Finally, we add the “Quit” button.

What’s the thing about “proportion”?

When you add several widgets to a sizer, the proportion is used to define the percentage of the space that each widget should take up. For example, when you add three widgets with proportions 5, 4, and 7, this is the space they’ll take up:

Sizer Proportions
horz_sizer = wx.Sizer(wx.HORIZONTAL)
horz_sizer.Add(btn_1,
               proportion=5)
horz_sizer.Add(btn_2,
               proportion=4)
horz_sizer.Add(btn_3,
               proportion=7)

Proportion 0 means “use the minimum required space for the widget.”

Next, we create a vertical sizer, add the horizontal sizer with the buttons and the text box, and set the sizer to the frame:

vert_sizer = wx.BoxSizer(wx.VERTICAL)
        vert_sizer.Add(horz_sizer, proportion=0,
                       flag=wx.EXPAND | wx.ALL, border=4)
        vert_sizer.Add(text_box, proportion=1, flag=wx.EXPAND)

        self.SetSizer(vert_sizer)
        self.Fit()

What we see:

  • We specify the “wx.EXPAND” flag. This will be explained shortly.
  • The “wx.ALL” flag specifies that the border should apply to all sides (it’s a shortcut for “wx.LEFT | wx.TOP | wx.RIGHT | wx.BOTTOM”).

What does “wx.EXPAND” do?

Sizer EXPAND

Without “wx.EXPAND”, a horizontal sizer aligns the widgets in columns, but it does not touch their heights. Similarly, a vertical sizer aligns the widgets in rows, but it does not touch their widhts.

When “wx.EXPAND” is specified when adding a widget to a horizontal sizer, the sizer will adjust the height of the widget to the height of the sizer. (The height of the sizer is the maximum height of its children.) Similarly, “wx.EXPAND” tells a vertical sizer to adjust the width of the widget.

Tip: When the sizers do not work as expected, the Widget Inspector might help you find the problem. Select a widget and click the “Highlight” button to check whether it takes up the space that you expected.

Getting Rid of the Console

When you double-click a .py file, a console window opens. This is annoying and useless for GUI applications.

This can be solved in two ways:

  • Rename the .py file to .pyw
  • Run the .py file with “pythonw.exe” instead of “python.exe”

If you still want the output of “print” statements to be visible, pass “redirect=True” to the initializer of the “wx.App” object. A separate window will be opened when a “print” occurs.

Advanced: You can also write your own file-like object (like “StringIO”) that you assign to “sys.stdout” and “sys.stderr” and that appends all texts to a “Log Messages” window in the GUI.

GUI Tools for Creating GUIs

There are tools that let you layout frames and dialog boxes graphically. Personally, I prefer creating widgets programmatically, because the graphical editors that I tried all have shortcomings. If you want to check for yourself, see XRCed, which comes with the wxPython Demos and Tools.

Homework

Run this command in the Python shell:

>>> import this

Advertisement

Leave a Reply

Your email address will not be published. Required fields are marked *