PyGX: A Bird View¶

Welcome to PyGX! If you are new to PyGX, this notebook is for you. It typically takes 5-10 minutes to complete, with code examples. By the end of this read, you'll have a brief understanding of what PyGX is, when it is useful and how to use it, in particular:

• Symbolic object-oriented programming (SOOP): You'll learn how to develop symbolic classes effectively and explore important concepts like operations, events, markers, and detour.

• PyGX's applications: We'll provide an overview of the various applications of PyGX, with a particular focus on program search (e.g. evolutionary computing, AutoML).

With these knowledge, you'll be well-prepared to delve deeper into PyGX and its extensive features, allowing you utilize it effectively in a variety of programming contexts.

In [ ]:

!pip install pygx

!pip install pygx

SOOP: PyGX's Foundation¶

PyGX introduces symbolic object-oriented programming (SOOP), built on the concept of a symbolic object: an object you can use as usual, and also manipulate through the representation it was built from. Symbolic objects are instances of symbolic classes — here we use a simple Foo class to illustrate the basics.

Developing Symbolic Classes¶

First, a symbolic class is created by subclassing pg.Object, with its fields declared as annotations. topo=True additionally places instances in a symbolic tree, which the events section below relies on:

In [2]:

from typing import Any
import pygx as pg

# `topo=True` opts into the symbolic tree: `topo_parent`, `topo_path`
# and the topology events used later in this notebook.
class Foo(pg.Object, topo=True):
  # `Any`, so later cells can place placeholders (TBD, pg.oneof) here.
  x: Any
  y: Any

  def on_sym_ready(self):
    # Runs at construction AND after every rebind, so `z` stays in step.
    super().on_sym_ready()
    self.z = self.x + self.y

  def hello(self):
    print('hello, pygx!')

from typing import Any import pygx as pg # `topo=True` opts into the symbolic tree: `topo_parent`, `topo_path` # and the topology events used later in this notebook. class Foo(pg.Object, topo=True): # `Any`, so later cells can place placeholders (TBD, pg.oneof) here. x: Any y: Any def on_sym_ready(self): # Runs at construction AND after every rebind, so `z` stays in step. super().on_sym_ready() self.z = self.x + self.y def hello(self): print('hello, pygx!')

Then we can create an instance of Foo, which is used just like any other object.

In [3]:

foo = Foo(x=1, y=2)
foo.hello()
foo.z

foo = Foo(x=1, y=2) foo.hello() foo.z

hello, pygx!

Out[3]:

3

Operations¶

A range of operations can be applied to symbolic objects. First, we can print the human-readable representation of the foo object:

In [4]:

foo

foo

Out[4]:

Foo(...)

Foo( x=1, y=2 )

xx

int

1

1

yy

int

2

2

foo is also mutable, the internal state z is automatically recomputed from the new x and the old y.

In [5]:

foo.sym_rebind(x=2)
foo.z

foo.sym_rebind(x=2) foo.z

Out[5]:

4

But where is y stored? From the class definition, it's not assigned to any member of foo. The answer is that y is stored as a symbolic attribute of foo, accessible via the sym_init_args property:

In [6]:

foo.sym_init_args

foo.sym_init_args

Out[6]:

{'x': 2, 'y': 2}

Moreover, we can query foo's representation:

In [7]:

pg.query(foo, where=lambda v: isinstance(v, int))

pg.query(foo, where=lambda v: isinstance(v, int))

Out[7]:

{'x': 2, 'y': 2}

Another important aspect is symbolic equality, which is critical for comparing whether two concepts are the same. The comparison is based on neither reference nor internal state, but solely on the representations:

In [8]:

print(pg.eq(foo, Foo(x=2, y=2)))
print(pg.eq(foo, Foo(x=2, y=3)))

print(pg.eq(foo, Foo(x=2, y=2))) print(pg.eq(foo, Foo(x=2, y=3)))

True
False

Similarly, we can compare two symbolic objects:

In [9]:

print(pg.lt(foo, Foo(x=2, y=3)))
print(pg.lt(foo, Foo(x=0, y=3)))

print(pg.lt(foo, Foo(x=2, y=3))) print(pg.lt(foo, Foo(x=0, y=3)))

True
False

We can also get the symbolic hash, which is computed solely from the representation:

In [10]:

print(pg.hash(foo))
print(pg.hash(Foo(x=2, y=2)))

print(pg.hash(foo)) print(pg.hash(Foo(x=2, y=2)))

4983040805971298028
4983040805971298028

Moreover, foo can be cloned:

In [11]:

foo2 = foo.sym_clone()
print(foo2 != foo)
print(pg.eq(foo2, foo))

foo2 = foo.sym_clone() print(foo2 != foo) print(pg.eq(foo2, foo))

False
True

It can also be serialized and deserialized:

In [12]:

json_str = foo.to_json_str()
print(json_str)

foo3 = pg.from_json_str(json_str)
print(pg.eq(foo3, foo))

json_str = foo.to_json_str() print(json_str) foo3 = pg.from_json_str(json_str) print(pg.eq(foo3, foo))

{"_type": "__main__.Foo", "x": 2, "y": 2}
True

Validation¶

Next, let's look at validation. What if we want to limit x and y to positive integers? We can attach a value spec to a field with pg.field:

In [13]:

class Foo(pg.Object, topo=True):
  x: int = pg.field(value_spec=pg.typing.Int(min_value=1))
  y: int = pg.field(value_spec=pg.typing.Int(min_value=1))

  def on_sym_ready(self):
    super().on_sym_ready()
    self.z = self.x + self.y

class Foo(pg.Object, topo=True): x: int = pg.field(value_spec=pg.typing.Int(min_value=1)) y: int = pg.field(value_spec=pg.typing.Int(min_value=1)) def on_sym_ready(self): super().on_sym_ready() self.z = self.x + self.y

Then we can see the instantiation on Foo will fail since the value of x is not positive.

In [14]:

try:
  Foo(x=0, y=1)
except ValueError as e:
  print('Error', e)

try: Foo(x=0, y=1) except ValueError as e: print('Error', e)

Error Value 0 is out of range (min=1, max=None). (path=)

Value specifications can also be programmed:

In [15]:

def value_or_factory(value_spec):
  return pg.typing.Union([
      value_spec,
      pg.typing.Callable(returns=value_spec)
  ])

class Bar(pg.Object):
  x: Any = pg.field(
      value_spec=value_or_factory(pg.typing.Int(min_value=1)))

# Okay!
Bar(x=1)

# Okay too!
Bar(x=lambda: 1)

def value_or_factory(value_spec): return pg.typing.Union([ value_spec, pg.typing.Callable(returns=value_spec) ]) class Bar(pg.Object): x: Any = pg.field( value_spec=value_or_factory(pg.typing.Int(min_value=1))) # Okay! Bar(x=1) # Okay too! Bar(x=lambda: 1)

Out[15]:

Bar(...)

Bar( x=<function <lambda> at 0x113cf22a0> )

xx

function(...)

<function <lambda> at 0x113cf22a0>

<function <lambda> at 0x113cf22a0>

Events¶

A symbolic object is also aware of its containing object and its position in the tree:

In [16]:

foo = pg.from_json_str(json_str)
print(foo.topo_parent)
print(foo.topo_path)

# When `foo` is assigned to a symbolic dict,
# its `topo_parent` property get updated.
d = pg.Dict(x=foo)
print(foo.topo_parent)
print(foo.topo_path)

# When the parent dict reassign the key to another
# value, `foo.topo_parent` get updated again.
d.x = 1
print(foo.topo_parent)
print(foo.topo_path)

foo = pg.from_json_str(json_str) print(foo.topo_parent) print(foo.topo_path) # When `foo` is assigned to a symbolic dict, # its `topo_parent` property get updated. d = pg.Dict(x=foo) print(foo.topo_parent) print(foo.topo_path) # When the parent dict reassign the key to another # value, `foo.topo_parent` get updated again. d.x = 1 print(foo.topo_parent) print(foo.topo_path)

None

{
  x = Foo(
    x = 2,
    y = 2
  )
}
x
None

We can also subscribe to such events:

In [17]:

class Foo(pg.Object, topo=True):
  x: int = pg.field(value_spec=pg.typing.Int(min_value=1))
  y: int = pg.field(value_spec=pg.typing.Int(min_value=1))

  def on_sym_ready(self):
    super().on_sym_ready()
    self.z = self.x + self.y

  def on_topo_parent_change(self, old_parent, new_parent):
    super().on_topo_parent_change(old_parent, new_parent)
    print('Parent has changed from %r to %r'
          % (old_parent, new_parent))

  def on_topo_path_change(self, old_path, new_path):
    super().on_topo_path_change(old_path, new_path)
    print('Path has changed from %r to %r'
          % (old_path, new_path))

f = Foo(x=1, y=2)
d = pg.Dict(x=f)
d.x = 1

class Foo(pg.Object, topo=True): x: int = pg.field(value_spec=pg.typing.Int(min_value=1)) y: int = pg.field(value_spec=pg.typing.Int(min_value=1)) def on_sym_ready(self): super().on_sym_ready() self.z = self.x + self.y def on_topo_parent_change(self, old_parent, new_parent): super().on_topo_parent_change(old_parent, new_parent) print('Parent has changed from %r to %r' % (old_parent, new_parent)) def on_topo_path_change(self, old_path, new_path): super().on_topo_path_change(old_path, new_path) print('Path has changed from %r to %r' % (old_path, new_path)) f = Foo(x=1, y=2) d = pg.Dict(x=f) d.x = 1

Path has changed from  to x
Parent has changed from None to {}
Parent has changed from {x=Foo(x=1, y=2)} to None
Path has changed from x to 

Functors¶

A functor is a function whose call is a symbolic object: you can bind its arguments now, inspect and edit them, and invoke it later. Create one with the pg.functor decorator:

In [18]:

@pg.functor
def foo(x, y):
  return x + y

@pg.functor def foo(x, y): return x + y

Calling foo does not run it — it returns a symbolic object, since the functor is a class with a __call__ method:

In [19]:

# `f` is a symbolic object representing a bound
# call to the functor `foo`.
f = foo(1, 2)
print(f)

# It can be invoked separately.
f()

# `f` is a symbolic object representing a bound # call to the functor `foo`. f = foo(1, 2) print(f) # It can be invoked separately. f()

foo(
  x = 1,
  y = 2
)

Out[19]:

3

Besides the capabilities inherited from symbolic classes, functors are powerful in their advanced binding:

In [20]:

# Partial binding with x=1:
f = foo(1)
print(f)

# Call `f` with `y` specified at call time,
# however, `y` is not bound with `f`.
print(f(y=1))
print(f.y)

# Incrementally bind y=2:
f.y = 2
print(f)

# Rebind x to 2:
f.sym_rebind(x=2)
print(f)
print(f())

# Override args at call time:
f(x=3, override_args=True)

# Partial binding with x=1: f = foo(1) print(f) # Call `f` with `y` specified at call time, # however, `y` is not bound with `f`. print(f(y=1)) print(f.y) # Incrementally bind y=2: f.y = 2 print(f) # Rebind x to 2: f.sym_rebind(x=2) print(f) print(f()) # Override args at call time: f(x=3, override_args=True)

foo(
  x = 1,
  y = MISSING_VALUE
)
2
MISSING_VALUE
foo(
  x = 1,
  y = 2
)
foo(
  x = 2,
  y = 2
)
4

Out[20]:

5

Markers¶

Parts of a symbolic object can be marked with pure symbolic values, which stand for a high-level concept that is not yet materialized for execution:

In [21]:

# `topo=True` to match `Foo`: a child placed in a symbolic tree
# has its path set by the parent.
class TBD(pg.symbolic.PureSymbolic, pg.Object, topo=True):
  pass

foo = Foo(x=TBD(), y=TBD())
print(foo)
print(pg.is_abstract(foo))

# `topo=True` to match `Foo`: a child placed in a symbolic tree # has its path set by the parent. class TBD(pg.symbolic.PureSymbolic, pg.Object, topo=True): pass foo = Foo(x=TBD(), y=TBD()) print(foo) print(pg.is_abstract(foo))

Foo(
  x = TBD(),
  y = TBD()
)
True

In [22]:

try:
  # `foo` is abstract, so `Foo.__init__` is not yet
  # called on `foo`.
  foo.z
except AttributeError as e:
  print('Error', e)

try: # `foo` is abstract, so `Foo.__init__` is not yet # called on `foo`. foo.z except AttributeError as e: print('Error', e)

Error 'Foo' object has no attribute 'z'

Symbolic Detour¶

Sometimes the objects we care about are not exposed as function or class arguments:

In [23]:

class A:

  def __init__(self, x, y):
    self.x = x
    self.y = y

  def __call__(self):
    return self.x + self.y

def my_fun():
  return A(1, 2)()

my_fun()

class A: def __init__(self, x, y): self.x = x self.y = y def __call__(self): return self.x + self.y def my_fun(): return A(1, 2)() my_fun()

Out[23]:

3

Can we modify my_fun's behavior without touching its source code? Symbolic detour is the tool for that:

In [24]:

class B(A):

  def __call__(self):
    return self.x * self.y

# Within this context manager,
# A.__new__ is redirected to B.__new__.\
# NOTE: `pg.detour` can detour Python
# classes that are even not symbolized.
# However, it cannot be used to detour
# builtins.
with pg.detour([(A, B)]):
  print(my_fun())

# Out of the context manager,
# A.__new__ remains its original definition.
print(my_fun())  

class B(A): def __call__(self): return self.x * self.y # Within this context manager, # A.__new__ is redirected to B.__new__.\ # NOTE: `pg.detour` can detour Python # classes that are even not symbolized. # However, it cannot be used to detour # builtins. with pg.detour([(A, B)]): print(my_fun()) # Out of the context manager, # A.__new__ remains its original definition. print(my_fun())

2
3

PyGX's Applications¶

Program Search¶

Based on markers, PyGX provides a few pure symbolic classes called hyper primitives, which are helpful in representing a space of objects:

In [25]:

# A space of `Foo` with `x` chosen from [1, 2] and a fixed `y`.
foo_space = Foo(x=pg.oneof([1, 2]), y=3)

# We can inspect the space by looking at its genome shape:
print(pg.dna_spec(foo_space))

# Or looking at the size of the space:
print(pg.dna_spec(foo_space).space_size)

# A space of `Foo` with `x` chosen from [1, 2] and a fixed `y`. foo_space = Foo(x=pg.oneof([1, 2]), y=3) # We can inspect the space by looking at its genome shape: print(pg.dna_spec(foo_space)) # Or looking at the size of the space: print(pg.dna_spec(foo_space).space_size)

Space({
  0 = 'x': Choices(num_choices=1, [
    (0): 1
    (1): 2
  ])
})
2

For foo_space to be materialized, we need to provide the choice for x:

In [26]:

# Providing a genome to the space so it can be converted to
# a sample in the space:
foo = pg.materialize(foo_space, pg.DNA(0))
print(foo)

# Providing a genome to the space so it can be converted to # a sample in the space: foo = pg.materialize(foo_space, pg.DNA(0)) print(foo)

Foo(
  x = 1,
  y = 3
)

We can also use hyper primitives to generate samples from the space:

In [27]:

# Generate all possible examples from the space.
foo_space = Foo(x=pg.oneof([1, 2]), y=pg.oneof([3, 4]))
print(list(pg.iter(foo_space)))

# Random sample 3 examples:
print([x for x in pg.random_sample(foo_space, num_examples=3, seed=1)])

# Generate all possible examples from the space. foo_space = Foo(x=pg.oneof([1, 2]), y=pg.oneof([3, 4])) print(list(pg.iter(foo_space))) # Random sample 3 examples: print([x for x in pg.random_sample(foo_space, num_examples=3, seed=1)])

[Foo(x=1, y=3), Foo(x=1, y=4), Foo(x=2, y=3), Foo(x=2, y=4)]
[Foo(x=1, y=3), Foo(x=2, y=3), Foo(x=2, y=4)]

Two separate pg.oneof expressions are two independent decisions — above, x and y vary separately, giving 4 programs. When two fields must move together, bind one primitive and place it at both: a primitive reused at several locations is a single decision that lands everywhere.

In [28]:

class Tied(pg.Object, topo=True):
  x: Any
  y: Any

# One primitive, two locations -> one decision. Under `topo=True` a node holds
# one tree position, so the second location is spelled `pg.Ref`.
units = pg.oneof([1, 2])
tied_space = Tied(x=units, y=pg.Ref(units))

print('space size:', pg.dna_spec(tied_space).space_size, '(not 4)')
for t in pg.iter(tied_space):
  print(f'  x={t.x} y={t.y}')

# `pg.hyper.reference` covers what reuse cannot: following a value that is not
# itself a hyper primitive.
followed = Tied(x=pg.oneof([1, 2]), y=pg.hyper.reference('x'))
print('via reference:', [(t.x, t.y) for t in pg.iter(followed)])

class Tied(pg.Object, topo=True): x: Any y: Any # One primitive, two locations -> one decision. Under `topo=True` a node holds # one tree position, so the second location is spelled `pg.Ref`. units = pg.oneof([1, 2]) tied_space = Tied(x=units, y=pg.Ref(units)) print('space size:', pg.dna_spec(tied_space).space_size, '(not 4)') for t in pg.iter(tied_space): print(f' x={t.x} y={t.y}') # `pg.hyper.reference` covers what reuse cannot: following a value that is not # itself a hyper primitive. followed = Tied(x=pg.oneof([1, 2]), y=pg.hyper.reference('x')) print('via reference:', [(t.x, t.y) for t in pg.iter(followed)])

space size: 2 (not 4)
  x=1 y=1
  x=2 y=2
via reference: [(1, 1), (2, 2)]

Based on the search space, we can evolve a Foo to maximize the evaluations:

In [29]:

# Create a larger search space.
foo_space = Foo(x=pg.oneof(range(1, 101)), y=pg.oneof(range(1, 101)))
print('Space size', pg.dna_spec(foo_space).space_size)

# Use regularized evolution as search algorithm.
# Please refer to https://arxiv.org/abs/1802.01548
search_algorithm = pg.algo.evolution.regularized_evolution(
    pg.algo.evolution.mutators.Uniform(),
    population_size=100,
    tournament_size=20)

# Perform the search, using `z` (x + y) as the reward.
rewards = []
for foo, feedback in pg.sample(
    foo_space, search_algorithm, num_examples=500):
  reward = foo.z
  feedback(reward)
  rewards.append(reward)

# Create a larger search space. foo_space = Foo(x=pg.oneof(range(1, 101)), y=pg.oneof(range(1, 101))) print('Space size', pg.dna_spec(foo_space).space_size) # Use regularized evolution as search algorithm. # Please refer to https://arxiv.org/abs/1802.01548 search_algorithm = pg.algo.evolution.regularized_evolution( pg.algo.evolution.mutators.Uniform(), population_size=100, tournament_size=20) # Perform the search, using `z` (x + y) as the reward. rewards = [] for foo, feedback in pg.sample( foo_space, search_algorithm, num_examples=500): reward = foo.z feedback(reward) rewards.append(reward)

Space size 10000

In [30]:

# Plot the reward over time:
import matplotlib.pyplot as plt
plt.plot(range(len(rewards)), rewards)
plt.show()

# Plot the reward over time: import matplotlib.pyplot as plt plt.plot(range(len(rewards)), rewards) plt.show()

Programs that Need Lots of Flexiblity¶

SOOP offers a high degree of flexibility, thanks to the mutability of symbolic objects. This enables partial and late binding, and grants access to the power of rebinding. Program search is just one example. Other use cases, like flexible function binding and direct manipulation, show the same versatility. For more on these, see Symbolic Functions and Interactive SVG.

Representation-centric Systems¶

Symbolic objects inherently serve as representations, which simplifies representation-centric systems. Machine learning systems are a prime example: they typically rely on a separate configuration system, which is no longer required with PyGX. For further details on PyGX's application in machine learning, please refer to the Symbolic Machine Learning examples.

More¶

PyGX offers a wide range of capabilities for you to discover, such as the ability to create domain-specific languages using markers and develop context-aware components that can seamlessly adjust to changes in object hierarchy. We are excited to see the innovative ways in which you utilize SOOP and PyGX and look forward to learning from you.