Seminar 19 Context, settings, and exceptions

Today, we will reuse our lovely Guitar Hero game and see how we simplify our life via having settings in a separate file. Plus we will be using context to isolate repetitive initialization / clean-up code and exceptions to control the flow. Thus, create a new folder with all the classes and the final exercise from Guitar Hero, which you will serve as a foundation for exercise 1.

19.1 Context manager

On the one hand, context management is a frequently used feature in Python, particularly for file operations. On the other hand, its full power is rarely is less known even though it can be very useful whenever you the context of your program is the same or very similar, as in case of the PsychoPy games that we programmed and typical PsychoPy experiments. In both cases, there is a fairly fixed structure of the program:

  1. Initialization
    • define experimental settings by either reading them from an external file, as you will learn below, or creating constants (less flexible but what we have been doing so far)
    • create PsychoPy window, logger for experimental results, mouse (if required), initialize special devices such as response box, eye tracker, etc.
  2. Actual experiment
  3. Saving and cleaning up
    • save data logs
    • if required, close connection to special devices such as response boxes, eye tracker, etc.
    • close PsychoPy window

If you look at your code, you will realized that steps 1 and 3 remain fairly the same throughout all the games that we programmed. Thus, we will create a context manager class, which you can always reuse, that will hide away the boilerplate code.

Here is an example of how use of a context manager helps when working with files (something, we will need to use later for our settings files). First, how it works without a context manager: you open a file and assign the object to variable, you work with it, you close it. The latter is important to ensure that information was fully written into it and that you do not lock for file.

file = open("somefile.txt", "r")
# ... do something with the file, such as reading the entire file into a single variable
data = file.read()
close(file)

A better way is to use a context manager that file class implements via with ... as ... statement.

with open("somefile.txt", "r") as file:
    file.read()

Note that now the file.read() is inside of the with block and there is no file.close() call. The latter is evoked automatically, once you run all the code inside the with block and exit it. Although for this example the difference is minimal — a different way to assign a value to a variable and explicit vs. implicit file closing — the second variant takes care of cleaning up, ensures that you do not forget about it, and allows you to concentrate on the important bits.

Here is how it works behind the scenes. A context manager is a class that implements two special methods __enter__ and __exit__ (so, this is a classic Python duck typing). The former creates and returns a context, which is whatever attribute or value you require, wheres the former performs cleaning up that is necessary before exiting the context. Here is how we would implement a limited file context manager by ourselves:

class FileManager():
    def __init__(self, filename, mode):
        """
        Stores the settings for use in __enter__
        
        Parameters
        ----------
        filename : str
        mode : str
        """
        self.file = None
        self.filename = filename
        self.mode = mode
        
    def __enter__(self):
      """ 
      What we need to do to create context:
        * Open the file and returns the object.
      
      Returns
      ----------
      File object
      """
      self.file = open(self.filename, self.mode)
      return self.file
      
    def __exit__(self, exc_type, exc_value, traceback):
      """
      What we need to do before destroying the context:
        * Close the file before we exit the context.
      """
      close(self.file)
      
# and now we use it!
with FileManager("somefile.txt", "r") as file:
    file.read()

Note that __exit__ method has extra parameters exc_type, exc_value, and traceback. They will be relevant for exception handling later on but for now, you can ignore them.

Now is your turn! Create a WindowContext class (in a separate file, of course) that will create a PsychoPy Window object of a given size, which is passed to the constructor, and closes when the code exists the context. For now, you will need one attribute to store information about the requested size and one attribute for PsychoPy Window itself (hint, use winas an attribute name). There will be a small but important difference relative to FileManager class in the example: There will be other objects that are context-relevant (settings, logging class) that we will add later on, so instead of returning just the window object, the enter it should return the reference to the object itself (reminder, reference to the current object is always in the self parameter of a method). Then, you can use it to create a 640-by-480 window (use whatever window size you need of course).

with WindowContext((640, 480)) as ctx:
  # your usual code inside but
  # PsychoPy window is ctx.win
  ctx.win.flip()

Create WindowContext class and use it in exercise01.py.

19.2 Settings file formats

So far, we either hard-coded specific values or defined them as constants (a better of these two approaches). However, this means that if you want to run your game with different settings, you need to modify the program itself. And if you want to have two versions of the game (two experimental conditions), you would need to have two programs with all the problems of maintaining virtually identical code in several places at once.

A better approach is to have separate files with settings, so you can keep the code constant and alter specific parameters by specifying which settings file the program should use. This is helpful even you plan to have a single set of setting as it separates code from constants, puts the latter all in one place and makes easier to edit and check them. There are multiple formats for settings files: XML, INI, JSON, YALM, etc.

19.2.1 XML

XML — an Extensible Markup Language — looks similar to HTML (HyperText Markup Language). Experiments designed using PsychoPy Builder interface are stored using XML files but with .psyexp extension. A settings file for Guitar Hero in XML could look like this

<Target>
  <width>0.4</width>
  <height>0.1</height>
  <locations>
    <location>-0.5</location>
    <location>0</location>
    <location>0.5</location>
  </locations>
</Target>

The advantage of XML is that it is very flexible yet structured and you can use native Python interface to work with them. However, XML is not easy for humans to read, it is overpowered for our purposes of having a simple set of unique constants and its power means that using it is fairly cumbersome.

from xml.dom import minidom
settings = minidom.parse('test.xml')
settings.getElementsByTagName("target")[0].getElementsByTagName("width")[0].firstChild.data # this will give you string "0.4"

19.2.2 INI

This is a format with a structure similar to that found in MS Windows INI files.

[Target]
width = 0.4
height = 0.1
locations = -0.5, 0, 0.5

As you can see it is easier to read and Python has a special configparser library to work with them. The object you get is, effectively, a dictionary with additional methods and attributes. Note, however, that ConfigParser does not try to guess the type of data, so all values are stored as strings and it is your job to convert them to whatever type you need, e.g. integer, list, etc.

import configparser
settings = configparser.ConfigParser()
settings.read('settings.ini')
settings["Target"]["width"] # this will give you a string '0.4'

19.2.3 JSON

JSON (JavaScript Object Notation) is a popular format for web applications that use it to exchange data between server and client.

{
  "Target": {
    "width": 0.4,
    "height": 0.1,
    "locations": [-0.5, 0, 0.5]
  }
}

You can parse any string with JSON format into a dictionary in Python using json module. Its advantage over INI files is that JSON explicitly specifies data type, so it converts it automatically. Note that unlike configparse, json module does not work with files direction, so you need to open it manually.

import json
with open('settings.json') as json_file:
    settings = json.load(json_file)
    
settings["Target"]["width"] # this will give a float number 0.4

19.2.4 YAML

YAML (YAML Ain’t Markup Language, rhymes with camel) is very similar to JSON but its config files are more human-readable. It has fewer special symbols and brackets but, as in Python, you must watch the indentations as they determine the hierarchy.

Target:
  width : 0.4
  height : 0.1
  locations : [-0.5, 0, 0.5]

You will need to install a third-party library pyyaml to work with YAML files. Otherwise, you get the same dictionary as for the JSON

import yaml
with open("settings.yaml") as yaml_stream:
    settings = yaml.load(yaml_stream)
    
settings["Target"]["width"] # this will give a float number 0.4

19.3 Settings file for Guitar Hero

My personal preference is for yaml. However, it requires a separate library, so you could use json file instead. Create a settings file and call it settings.json (or use your imagination and come up with a better name). You will add more settings later on but for now just define a window size as

{
  "window": {
    "size" : [640, 480]
  }
}

In exercise02.py simply read the settings into a dictionary and print it out the window size. Remember that you will get a dictionary of dictionaries and think about keys that you need to access that size entry. Check that there are no errors and that dictionary has the correct structure. There should be no further code in exercise02.py.

Implement reading and printing settings in exercise02.py.

19.4 Context based on settings

Now, we will settings when we create the context. Create a new context manager class SettingsContext that will take a string with settings file name as the only input. In the __enter__, you should read settings into settings attribute (reuse your code from exercise 2) and then use window size from that settings to create PsychoPy window and store it in a win attribute. Again, you return the reference to an object itself. For now, the only thing you need to do in the __exit__ is to close the window. Use it the same way as before but for the file name instead of the size.

with SettingsContext('settings.json') as ctx:
    # your usual code inside but
    # PsychoPy window is ctx.win
    ctx.win.flip()

Create SettingsContext class and use it in exercise03.py (based on exercise01.py, what do you need to change?).

19.5 Using settings

The next exercise is simple: Look through your code, determine which values, either explicit constants or hard-coded ones, are effectively settings, define them in the settings.json file (create a nice hierarchical structure for them), and use them in the code. Hint, if you can get all settings for a window via ctx.settings["Window"]. Think how you can use this to simplify passing multiple settings to Target and TimedResponseTask.

Update Target and TimedResponseTask classes, put updated main script into exercise04.py.

19.6 Logging data

In the real experiment, we would want to log the responses, e.g., when participants pressed a button and what was the score. PsychoPy provides a convenience class ExperimentHandler for such purposes that we already met when programming the Memory Game.

Reuse the code of SettingsContext class for a GameContext class. In addition to the original functionality, it should create an ExperimentHandler object as a data attribute, so you could use it as ctx.data in the main script. For the context __exit__, you should save the results as wide text. I suggest that the log file name should be one of the settings. In the main code, log the pressed key, time of press (relative to the game start), and the score of that press.

Create GameContext class, use in an updated main script in exercise05.py.

19.7 Exceptions

When you are running an actual experiment, one of the worries that you have is “what happens to the data I have already logged, program crashes with an error”? Not collecting a full measurement is bad but not keeping at least partial log is even worse, as you can still use it for analysis or as a guidance for future adjustments. Python, as other languages, has special mechanisms to handle exceptions that arise during the code execution.

Whenever an error occurs at a run time, it raises an exception: it creates an object of a special class that contains information describe the problem. For example, a ZeroDivisionError is raised whenever you try to divide by zero, e.g., 1 / 0 (you can try this in a Jupyter notebook). A KeyError is raised, if you using a dictionary with a wrong key, the code below will raise it:

a_dict = {"a_key" : 1}
a_dict["b_key"]

Similarly, an IndexError is raise, if you try to use an invalid index for a list, a NameError, if you are trying to access variable that does not exist, AttributeError when an object does not have an attribute you are trying to use, etc.

In Python, you can use try: ... except:...finally: operators:

try:
    # some code that might generate a runtime error
except:  
    # code that is executed if something bad happens
finally:
    # code that is executed both with and without exception
    
# code that is executed ONLY if there were no exceptions or if an exception was handled

In the simplest case, you need just the first two operators: try and except. Create a Jupyter notebook (that you will submit as part of the assignment) and write the code that generates a division-by-zero error but is handled via try...except.... In the except simply print out a message, so that you know that it was executed. Create another cell, copy the code and now check that the exception handling code is not executed, if the error is not generated (i.e., divide by some non-zero number).

Using except: catches all exceptions. However, you can be more specific and handle exceptions based on their class.

try:
    # some code that might generate a runtime error
except KeyError as key_error:  
    # code that is executed only if KeyError exception was raised
except ZeroDivisionError as zero_division_error:  
    # code that is executed only if ZeroDivisionError exception was raised
except:
    # code that is executed if any OTHER exception is raised.

Implement handling for KeyError and ZeroDivisionError, they should print out different messages to check that it works. Test it generating these runtime errors with your code. What happens if you have the first two specific exception handlers but no general except:?

So far, you generated exception but cause the errors in the code but you can raise these exceptions yourself via raise operator. For example, instead of dividing by zero, you can raise ZeroDivisionError() (note that you are create an object of the class, hence the round brackets!). Use it with you previous code, instead of an actual division by zero. Try raising other exception and see how your code handles them.

Put exception handling code is cell of a Jupyter notebook.

So far I have talked about exceptions as a way to report runtime errors. However, they can be used in a more general way to control the execution flow. You already did it when implementing class-as-an-iterator approach. Once you ran out of items to return, your nextmethod raised (StopIteration)[https://docs.python.org/3/library/exceptions.html#StopIteration] exception that informed thefor` loop that it has no more items to loop through. We will use that side of exception in the next section when dealing with context.

19.8 Exception within context

try..except... operators provide a general mechanism for exceptions handling but what happens if an exception is raised inside a context? You can, of course, put a try...except... in the code itself, something you should do, if you are planning to handling specific exceptions. However, if an exception occur in the code inside the context, Python will first exit the context, i.e., the __exit__ method. Moreover, it will put the exception information into the parameters exc_type (a class of the exception) and exc_value (an object of that class). This way, you can perform a proper clean-up (save data, close window, etc.) and then handle an exception or leave it alone, so that it propagates further and can be handled by other pieces of your code or stop the execution.

Here, we will use this mechanism not only for safe clean-up but also to make aborting an experiment (or a game) easy. In previous game with many rounds, you had nested loops that made aborting a game via escape key press awkward. You had to check if in the inner loop, differentiate between a normal end-of-round and abort outside, etc. We can make out life much easier via a combination of a context manager and a custom exception. First, create a custom GameAbort class, which is a descendant of the Exception class. You do not need any code in it, even a constructor does not need to be redefined, so use pass statement for its body (you do need to have at least one line of code in the class, so pass will create a line but with no actual executable code). Next, you raise GameAbort(), if the player pressed escape key (do not forget to import GameAbort class, so you can use it in the main script). Finally, in the exit method of the GameContext manager, you should check whether exc_type is GameAbort (exc_type will be None, if no exception occurred) and, important(!), return True in that case:

def GameContext:
    ...
    def __exit__(self, exc_type, exc_value, traceback):
        ...
        if exc_type is GameAbort:
            return True

That last bit return True informs Python that you handled the exception and all is good (not need to propagate it further). Now, you can safely abort your experiment from any code location, inside nested loops, functions, etc. In all cases, the exception will be propagated until the __exit__ method, doing away with awkward extra checks.

Create GameAbort exception class, update GameContext class to handle it, use this in an updated main script in exercise06.py.