Python tricks



Entry: Getting back to python
Date: Thu Sep 12 11:08:32 EDT 2013

It's been a long time - about a decade.
Much has changed since then it seems.
Some things I find interesting:

- More functional features, i.e. 
  - copy instead of inplace operations
  - list comprehensions

Entry: Return the object please!
Date: Thu Sep 12 11:11:37 EDT 2013

list([1,2,3]).append(4) -> NoneType

In general it seems that most functions do this properly, allowing
dot-chaining of operations.  Why doesn't append return the object?

Answer: The functional append is the '+' operator.

Some more info here[1].


[1] http://docs.python.org/2/tutorial/datastructures.html#functional-programming-tools



Entry: List comprehensions
Date: Thu Sep 12 11:12:54 EDT 2013

>>> [x*2 for x in [1,2,3]]
[2, 4, 6]



Entry: Python web scraping
Date: Thu Sep 12 13:20:01 EDT 2013

Very interesting presentation by Asheesh Laroia[1]

mechanize
urllib
urllib2
request
lxml
BeatifulSoup


[1] https://www.youtube.com/watch?v=52wxGESwQSA


Entry: Python TCP console
Date: Thu Sep 12 14:17:39 EDT 2013

[1] http://code.saghul.net/index.php/2012/09/17/interactive-python-interpreter-over-tcp-for-tornado-applications/
[2] http://docs.python.org/2/library/code.html
[3] http://www.tornadoweb.org/en/stable/



Entry: PySide
Date: Thu Feb 13 15:14:23 CET 2014

Is it necessary to keep a reference?

Something the code below doesn't work if the object 'h' isn't saved
somewhere.  Does a bound method not keep a GC reference to its object?

  btn = QtGui.QPushButton('button')
  h = myHandler()   # instance
  self.hs.append(h) # need to keep a reference!
  btn.clicked.connect(m.handle)



Entry: DIY property based testing
Date: Sat Mar 29 10:02:45 EDT 2014

Interesting problem.

Is a type a sequence (generator) of values or a generator of a
sequence (generator) of values?

The extra indirection seems like a good idea from the concept level,
but because everything is random, it doesn't seem that it really
matters.

So this seems to work:

def a_range(min, max):
    while True:
        yield random.randint(min, max)

a_byte = a_range(0, 255)

def a_list(el_type, min=0, max=10):
    while True:
        len = random.randint(min, max)
        yield [el_type.__next__() for i in range(len)]

def a_tuple(*el_types):
    while True:
        yield tuple([t.__next__() for t in el_types])

def forall(n, f, *arg_types):
    for i,args in zip(range(n), a_tuple(*arg_types)):
        f(*args)



Entry: Adding a "phone home" option
Date: Mon Oct  6 19:23:38 CEST 2014

Problem: no inbound ssh allowed.
Solution: use an outbound ssh connection to connect to console.

Several options:

- operator types command to set up ssh tunnel to local ports.
  local ports run console

- operator types command to set up ssh connection to remote host
  ssh -> socat -> unix socket listen + socat unix socket client






For the console [1][2]:

import code
code.interact()

[1] http://stackoverflow.com/questions/4163964/python-is-it-possible-to-attach-a-console-into-a-running-process
[2] https://docs.python.org/2/library/code.html


Entry: Phone home
Date: Sun Oct 12 08:39:29 EDT 2014

I'd like to find a way for a python application to 'phone home',
i.e. attach to a remote management console.

I tried the combination of pexpect + ssh's command= + socat which
works well, but does not work on windows without cygwin installed.

This raises two questions:

 - On windows, how do you connect pipes to a program, like Git's
   ssh.exe (mingw).

 - Is there a simple way to set up a secure tcp connection not piping
   through ssh?


Let's use ssh to setup the tunnel and use sockets on localhost.


Entry: pexpect ssh support phone home
Date: Sun Oct 12 16:12:30 EDT 2014

import pexpect # http://pexpect.readthedocs.org/en/latest/
import os

class channel:
    def __init__(self):
        self.px = False
        self.connect()

    def connect(self):
        # the other end is a 'command=' line in .ssh/authorized_keys
        # starting socat on a unix socket.
        # to test this can be run here directly
        # command = "socat - UNIX-LISTEN:/tmp/channel.sock"
        command = "ssh -q -F python/support.config support"
        print("%s" % ['connect',command])
        self.px = pexpect.spawn(command)
        self.px.setecho(False)

    def next(self):
        self.px.expect('\r\n', timeout=60*60*24)
        return(self.px.before)

    def loop(self):
        while True:
            cmd = self.next().decode("utf8")
            print("%s" % ['command',cmd])
            val = eval(cmd)
            msg = '%s' % ['value',val]
            print(msg)
            self.px.sendline(msg)

    def start(self):
        while True:
            try:
                self.loop()
            except pexpect.EOF:
                msg = '%s' % ['error','EOF']
                print("%s" % msg)
                self.connect()
            except Exception as e:
                msg = '%s' % ['error',e]
                self.px.sendline(msg)
                print("%s" % msg)

## support.config is a ssh config file containing something like:
## support_key and support_key.pub are rsa keys
# Host support
# Port 443
# User supportuser
# Hostname support.enterprise.com
# PasswordAuthentication no
# IdentitiesOnly yes
# IdentityFile ./support_key