Jython: Dictionaries and list comprehension

I know I'm getting carried away with this list comprehension topic, but I'm finding it a lot of fun to see what else I can do with list comprehension. It also matches my other favorite topic (for another  day) of slicking and indexing. You will see some of it here. What I'm doing here is exploring more ways of doing list comprehension. I''m sure most of you know how to work with dictionaries.

Create a dictionary:

>>> D = {'a': 'aardvark', 'b': 'bear', 'c': 'cat', 'd': 'dog', 'e': 'elephant', 'f': 'fox', 'g': 'giraffe', 'h': 'hippo', 'i': 'insect'}

List comprehensions on a dictionary are a little more complicated than a list, but it's fairly easy.

>>> [x for x in D]
['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i']
>>> [x for x in D.values()]
['aardvark', 'bear', 'cat', 'dog', 'elephant', 'fox', 'giraffe', 'hippo', 'insect']

Edited later:
>>> [x for x in D.items()]
[('a', 'apple'), ('b', 'balloon'), ('c', 'cat'), ('d', 'dog'), ('e', 'elephant'), ('f', ['fox'])]

>>> [x for x in D.items() if x[1] == 'elephant']
[('e', 'elephant')] # In this case, it sliced by element. Use x[0] to scan the keys.


You can use either .items() or .iteritems() to get key, value pairs,

Back to original post:

Say I want to get the last letter of each item in my dictionary values:

>>> [x[-1:] for x in D.values()]
['k', 'r', 't', 'g', 't', 'x', 'e', 'o', 't']

How about filtering:

# Get the elements where the last letter matches 't'.
>>> [x for x in D.values() if x[-1:] == 't']
['cat', 'elephant', 'insect']

As you see, slicing comes in handy with coding list comprehensions.

An earlier post, I eexplored how the dir() function can really be useful. Here's a little expansion on that topic:

>>> import os
>>> dir(os)
>>> len(dir(os))
110

I realized you really don't need a .startswith() or .endswith() call in a list comprehension to find matching elements. Just slice away!


Filter by first letter:
>>> [x for x in dir(os) if x[0] == 'e']
['environ', 'errno', 'error', 'extsep']

Outside of a list comprehensiuon, you would have to do this to get the first letter of an element.

>>> dir(os)[41]
'environ'
>>> dir(os)[41][0]
'e'

In the list comprehension, I didn't have to. Why? Because x already indexes the list, so I'm slicing each element to return any string beginning with 'e'.

If you need to know the index numbers to process later, you could:

>>> [dir(os).index(x) for x in dir(os) if x[0] == 'e']
[41, 42, 43, 44]

>>> [(dir(os).index(x),x) for x in dir(os) if x[0] == 'e']
[(41, 'environ'), (42, 'errno'), (43, 'error'), (44, 'extsep')]

As I've mentioned before, thinking a list comprehension through is simply a matter of putting the result you want first, then building the list comprehension to reach your result, as you can see in the example above. Now you have index intergers for all the elements beginning with 'e' from the list.

It's not necessary to use the keyword print inside the interactive session, but if you put a list comprehension in a script and need to show your result, put print before the list comprehension (e.g. print [x for ...])

I know there are programmers out there who are smareter and more experienced than me. As you see, I'm a quick learner and I relish the opportunity to continue learning.

I'm not shoinw off at all. Instead, I view this as a way to prod you and your ideas, so you can achieve your own success and sense of satisfaction at having done it. I am limited in my hearing and eyesight, but it doesn't stop me from doing what I can and stay busy in what I really enjoy.

Until later, happy coding!

Jython: Comprehending list comprehensions

This time, I'd like to discuss list comprehension. This is another tool used in Python and Jython. It's a very concise way of stepping through a list or a string, and processing information. You can build list comprehensions from the very basic to a complicated expression.

In an earlier post, I demonstrated a list comprehension at work with the dir() function, in returning information.

Here is a basic list comprehendion and they are enclosed in square brackets:

[expression for variable in sequence [if test-expression]]

test-expression is optional.

Let's see it in action:

>>>[x for x in ['one', 'two', 'three']]
['one', 'two', 'three']

To make it clear in my mind, I think of the first expression as the result I want, and build a list comprehension from that. Like so:

>>> [x.upper() for x in ['one', 'two', 'three']]
['ONE', 'TWO', 'THREE']

How about this?

>>> [x.startswith('t') for x in ['one', 'two', 'thre']]
[False, True, True]

It even does boolean expressions.

>>> a = 'ABC'
>>> word = ['Apple', 'Bird', 'Cat']
>>> [(x, y) for x in a for y in word if y.startswith(x)]
[('A', 'Apple'), ('B', 'Bird'), ('C', 'Cat')]

The above is an example of a nested list comprehension with an if test-expression. You noticed the result was list of tuples. You can put the result into a variable and print only what you need.

>>> result = [(x, y) for x in a for y in word if y.startswith(x)]
>>> result[0][1]  <<< here I indexed the tuple
'Apple'

>>> result
[('A', 'Apple'), ('B', 'Bird'), ('C', 'Cat')]

Or this:

>>> [x for (x, y) in result]
['A', 'B', 'C']

List comprehension can process files.

>>> [x for x in open('/home/patrick/script2.py')]
['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n', '\n']

If you are using list comprehension in a script, and you want to show the results on the screen, just put the print before the list comprehension.

>>>print [x for x in open('/home/patrick/script2.py')]
['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n', '\n']

Can you make a complex list comprehension? You can, with some work.

I was working on the Java class GregorianCalendar. I wanted to see if my system could disply other languages. So, I used the getAvailableLocales() method. However, this method returned a list of objects, not strings. I wanted to search for a particular county, by searching a partial string, but coudln't as U got an error saying x was not in the list.

As a workaround, I had to convert it to a string to search and have it return the index of the list element.. So, my original loop went like this:

for x in range(len(locale)):
    t = str(locale[x])
    if t.endswith('US'):
        subloc.append(locale[x])

After 3 hours or so, I was able to build a list comprehension that worked exactly as my loop:
subloc = [locale[x[0]] for x in enumerate(map(str, locale)) if x[1].endswith('US')]

Can you see how I did it? I will discuss enumerate() another day.

Is this really the Pythonic way? Maybe not, but list comprehension, when successful does marvel at the conciseness and simplicity of this powerful language.


List comprehensions allow Python/Jython do the work for you. This minimizes development time and is a great benefit of this language.

If you would like to see the full script, email me at psykiatris@gmail.com.

Till next time. Happy coding!

Jython: Using dir() to find Java classes

Today's post is on how to use the dir() function to see what classes and methods are in a Java package.

The dir() function is a very powerful tool you can use to find out information about a Java package. I will demonstrate some useful hints to help you.

When working with Java, you had to use the Reflection class to find out what classes and methods were available. In Jython, it is a lot easier.

For my system, my Java packages are stored in the /usr/share/java/ folder. I use Ubuntu 16.04. So I picked the following package to work with: /usr/share/java/bcprov.jar. Any jar file will work if you don't have this particular package.

First, I took a quick peek at the jar file list, by typing (in a terminal): jar -tf /usr/share/java/bcprov.jar. jar is the app that willunpack a jar file. The -tf switch retrieves a list of files in the named package. So, my output was:

org/
org/bouncycastle/
org/bouncycastle/LICENSE.class
org/bouncycastle/apache/
org/bouncycastle/apache/bzip2/
org/bouncycastle/apache/bzip2/BZip2Constants.class
org/bouncycastle/apache/bzip2/CBZip2InputStream.class
org/bouncycastle/apache/bzip2/CBZip2OutputStream$1.class
org/bouncycastle/apache/bzip2/CBZip2OutputStream$StackElem.class
org/bouncycastle/apache/bzip2/CBZip2OutputStream.class
org/bouncycastle/apache/bzip2/CRC.class
org/bouncycastle/asn1/
org/bouncycastle/asn1/A
--------The rest omitted for space----------

As you see, this jar file contains a lot of classes and methods. It would take forever to go through and find what you want. The dir() built-in really comes in handy, as we'll see later.

Now that I know the top two levels of the package list, I can now open up my Jython interactive environment (for immediate results).

You must specify the filename along with the path, or it will not import. So, I entered this:

>>> import sys
>>> sys.path.append('/usr/share/java/bcprov.jar')


I'm ready to start importing the jar package. I start off with import org.bouncycastle. Then I type dir(org.bouncycastle), and I get:

>>> import org.bouncycastle
>>> dir(org.bouncycastle)
['LICENSE', '__name__', 'asn1', 'cert', 'cmc', 'cms', 'crypto', 'dvcs', 'eac', 'est', 'i18n', 'iana', 'jcajce', 'jce', 'math', 'mozilla', 'openssl', 'operator', 'pkcs', 'pkix', 'pqc', 'tsp', 'util', 'voms', 'x509']

You probably noticed that this list displayed above did not show the apache (we saw this in the partial file list shown earlier. That's okay. You can import specifically by typing import org.bouncycastle.apache. But let's do the math package. So, we would type import org.bouncycastle.math as math.

The AS keyword allows you to assign an alias to the item you just importedSp., . So, instead of tying 'org.bouncycastle (and so on), now you can do type dir(math) and see a list.

>>> import org.bouncycastle.math as math
>>> dir(math)
['Primes', '__name__', 'ec', 'field', 'raw']

You see four things under the .math sib[aclage. The names in lower case are other subpackages. You still need to type the qualified package tree to import. One way you can type it is: 'from org.bouncycastle.math import ec as ec' (without the quotes). What this really does is that it imports only ec whithout importing everything else. This will save on memory.

You notice Primes is capitalized. This is a class. Methods start with a lower case with the next word capitalized. You can import Primes the way we've been doing it, or you can assign it to a name, like this:

>>> primes = math.Primes

What would have happened if I called it with a parentheses, as if I'm instantiating a class?

>>> primes = math.Primes()
Traceback (most recent call last):
  File "", line 1, in
TypeError: can't instantiate abstract class (org.bouncycastle.math.Primes)

You would get either the above message, or one that says no constructor is set for that class. If you attempt to call a class with a () pair and you get this error, just try again without the () pair. Then what can you do?

>>> primes = math.Primes
>>> dir(primes)
['MROutput', 'SMALL_FACTOR_LIMIT', 'STOutput', '__class__', '__copy__', '__deepcopy__', '__delattr__', '__doc__', '__ensure_finalizer__', '__eq__', '__format__', '__getattribute__', '__hash__', '__init__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__subclasshook__', '__unicode__', 'class', 'enhancedMRProbablePrimeTest', 'equals', 'generateSTRandomPrime', 'getClass', 'hasAnySmallFactors', 'hashCode', 'isMRProbablePrime', 'isMRProbablePrimeToBase', 'notify', 'notifyAll', 'toString', 'wait']

You see all the attributes that are available to Primes. Now, you're getting somewhere! Of course, you may know that the attributes in ALL CAPS are constants.

>>> primes.SMALL_FACTOR_LIMIT
211

Methods, which require parentheses and sometimes 1 or more arguments. That can be researched later, when you review the Java documentation.

But, what if you do a dir() command and it returns a lot of information. What then? This is where list comprehension and indexing really come in handy.

Here I imported the math.ec sub-package:

>>> from org.bouncycastle.math import ec as ec
>>> dir(ec)
['AbstractECMultiplier', 'DoubleAddMultiplier', 'ECAlgorithms', 'ECConstants', 'ECCurve', 'ECFieldElement', 'ECMultiplier', 'ECPoint', 'ECPointMap', 'FixedPointCombMultiplier', 'FixedPointPreCompInfo', 'FixedPointUtil', 'GLVMultiplier', 'MixedNafR2LMultiplier', 'MontgomeryLadderMultiplier', 'NafL2RMultiplier', 'NafR2LMultiplier', 'PreCompInfo', 'ReferenceMultiplier', 'ScaleXPointMap', 'ScaleYPointMap', 'WNafL2RMultiplier', 'WNafPreCompInfo', 'WNafUtil', 'WTauNafMultiplier', 'WTauNafPreCompInfo', 'ZSignedDigitL2RMultiplier', 'ZSignedDigitR2LMultiplier', '__name__', 'custom', 'endo', 'tools']


I can type len(dir(ec) to see how many elements are in the list.

Say I only want to see items starting with 'M'. Here's list comprehension at work:


>>> [x for x in dir(ec) if x.startswith('M')]
['MixedNafR2LMultiplier', 'MontgomeryLadderMultiplier']

You can also index the dir(ec) list:

>>> dir(ec)[4]
'ECCurve'

What if you only want to see the first 10 items from the dir(ec) list? Slicing does the trick:

>>> dir (ec)[0:10]
['AbstractECMultiplier', 'DoubleAddMultiplier', 'ECAlgorithms', 'ECConstants', 'ECCurve', 'ECFieldElement', 'ECMultiplier', 'ECPoint', 'ECPointMap', 'FixedPointCombMultiplier']

So, now you know how the dir() function can help you in finding a class or wmethod within a jar package.

As I continue my journey of learning Jython/Python, I will be sharing what I've learned along the way. I really enjoy this language and its simplicity and powerful abilities. I hope you will too.

Until later, happy coding!