Marker

A regular Python object serves as a program state after its creation, so it must be constructed with all required arguments fully specified and meet the type definition. Symbolic objects, in contrast, can serve as pure representations and can exist before being fully specified. This enables developers to start with an unfinished representation and gradually make it concrete. This is achieved through symbolic markers, which result in abstract objects.

Abstract objects

Abstract objects are symbolic objects that are not concrete, meaning they are not yet ready for triggering the __init__ logic upon creation. For example, Add(x=TBD(), y=1) represents an addition between a to-be-determined value and 1. Abstract objects can be partial objects or pure symbolic objects.

Partial objects

Partial objects are objects that have missing parts, which can be instantiated through the class method partial. Under the hood, the missing parts in the symbolic object are marked with pg.MISSING_VALUE. Such marking can occur at the immediate-children level or deeper into sub-trees. For example:

class Exp(pg.Object):
    # `Any`, so the example can nest one `Exp` inside another below.
    x: Any
    y: Any

    def on_sym_ready(self):
        # Runs only once every field is present.
        super().on_sym_ready()
        print('`on_sym_ready` is called.')


# `a` is a partial object as `a.y` is not specified.
a = Exp.partial(x=1)
assert pg.is_partial(a)
assert a.sym_init_args['y'] == pg.MISSING_VALUE

# `b` is also a partial object as it contains partial object `a` as its sub-node.
b = Exp.partial(x=a, y=2)
assert pg.is_partial(b)

# Fill in the missing part. `y` is what was absent, so rebinding it is what
# makes `a` — and therefore `b`, which holds it — concrete.
# At this point, we see the message "`on_sym_ready` is called" printed.
a.sym_rebind(y=2)
assert not pg.is_partial(a)
assert not pg.is_partial(b)

More on partial-object creation

Partial objects need to be explicitly created with partial. For example:

# Raises: `y` is not provided.
Exp(x=1)

# Raises: `y` is partial.
Exp(x=1, y=Exp.partial(x=1))

This means that when users need to create a hierarchy of partial objects, every containing class needs to call partial explicitly. This prevents human errors, but is also inconvenient. PyGX offers the context manager pg.allow_partial for this scenario, allowing partial objects to be created using standard class constructors:

with pg.allow_partial():
    a = Exp(x=1, y=Exp(x=1))
assert pg.is_partial(a)

For a partial object, the missing values in the object hierarchy can be queried via sym_missing:

# Shall print {'y.y': pg.MISSING_VALUE}
a.sym_missing()

Partial functions

For functions, there is a distinction between a partially bound function and a partial function object.

A partially bound function is a pg.Functor object whose arguments are partially specified, but each specified argument is concrete. For example:

@pg.functor()
def foo(x, y):
    return x + y


@pg.functor()
def bar(a, b):
    return a() + b()


# `f` is partially bound, but not partial.
f = foo(1)
assert not f.is_fully_bound
assert not pg.is_partial(f)
# `f` can be evaluated by providing the missing argument at call time.
assert f(y=2) == 3

# `g` is not partial since `f` is not partial.
g = bar(f)
assert not pg.is_partial(g)

# Raises: calling `a()` within `bar` will fail since `f` is partially bound.
# However, it's the user's responsibility to ensure a partially bound function
# may be used as an argument.
g(b=foo(1, 2))

On the other hand, a partial function object is a pg.Functor object whose bound arguments contain partial values. For example:

class Foo(pg.Object):
    v: int

    def __call__(self):
        return self.v ** 2


# `f` is now partial since `Foo.partial()` is partial.
f = bar(Foo.partial())

Pure symbolic values

A pure symbolic value occupies a field for representation, not for execution. It is a stand-in: something that says what a slot means before anything has decided what goes in it. pg.oneof(['a', 'b', 'c']) is the familiar one — it does not evaluate to a letter, it represents the choice between them until a search algorithm settles it.

This is what separates symbolic OOP from ordinary OOP. An ordinary object must be built from values it can run with; here you can express the shape of an idea first and supply the values later, which decouples stating an idea from implementing it.

The leaves are instances of pg.PureSymbolic subclasses, and anything holding one is pure symbolic too — the property propagates up the tree:

class Foo(pg.Object):
    x: Any
    y: Any

    def on_sym_ready(self):
        super().on_sym_ready()
        self.z = self.x + self.y


class Bar(pg.Object):
    foo: Foo

    def __call__(self):
        return self.foo.x * self.foo.y


# `bar1` is a concrete object since all its sub-nodes are concrete.
bar1 = Bar(foo=Foo(x=1, y=2))
assert not pg.is_pure_symbolic(bar1)


class TBD(pg.PureSymbolic):
    pass


# `bar2` is pure symbolic since its `foo` argument is pure symbolic, which
# contains an object of `TBD` (a subclass of `PureSymbolic`).
bar2 = Bar(foo=Foo(x=TBD(), y=2))
assert pg.is_pure_symbolic(bar2)

Delayed evaluation

Because a pure symbolic value is representational, there is nothing to run yet. What is delayed is the reactive lifecycle: on_sym_ready — and any state it computes — waits until the object becomes concrete. Fields stay readable throughout (they hold the markers themselves).

# Fields are readable — `bar2.foo.x` is the `TBD` marker itself.
assert isinstance(bar2.foo.x, TBD)

# Raises AttributeError: `on_sym_ready` has not run, so `z` does not exist.
bar2.foo.z

# Manipulate `bar2` into a concrete object by replacing all `TBD` with
# integer 1, which triggers `on_sym_ready` down the tree.
bar2.sym_rebind(lambda k, v, p: 1 if isinstance(v, TBD) else v)

# Okay: `on_sym_ready` has run now that `bar2` is concrete.
assert bar2.foo.z == 3

# Okay: `bar2.__call__` works on the concrete values.
assert bar2() == 2

Methods are not guarded

The delay covers the lifecycle hooks, not ordinary methods: calling bar2() before the rebind above runs its body, which receives the marker object itself rather than a value (here failing with a TypeError from TBD * int). The same applies to a functor holding a pure symbolic argument. Whether that fails depends on what the body does with it:

@pg.functor()
def bar(v):
    return v * 2

space = bar(pg.oneof([1, 2]))
space()          # raises TypeError: unsupported operand type(s)
                 # for *: 'OneOf' and 'int'

A body that merely passes the argument along will not raise at all — it just hands back the OneOf, which is rarely what the caller wanted. As with partial functions, it is the caller's responsibility not to invoke one before it is concrete. Materialize first, then call:

for program in pg.iter(space):
    program()    # 2, then 4 — each `program` is concrete

(pg.iter yields one concrete bar per candidate, so the body finally sees an int.)

Marker targets

A PureSymbolic subclass developer can control what symbolic fields the current pure symbolic class can stand in for. For example, the hyper primitive pg.oneof makes sure all candidate values are acceptable to the target field when it is used as a marker. This is done by implementing the custom_apply method, which is inherited from the pg.typing.CustomTyping interface.

What a materialized program owns, and what it shares

Passing a value into a hyper primitive adopts it: it becomes a node in the search space, at a position inside the candidate list.

class Model(pg.Object, topo=True):
    units: int = 8


class Exp(pg.Object, topo=True):
    model: Any = None
    tok: Any = None


m = Model(units=8)
m.topo_path                     # '' — free-floating

space = Exp(model=pg.oneof([m, Model(units=16)]))
m.topo_path                     # 'model.candidates[0]' — now held by the space

Materializing a program therefore cannot hand m straight back: a node has one position, and the program needs it at model. So a topo=True candidate is copied on the way out — the materialized program is independent of the space that described it:

exp = pg.materialize(space, pg.DNA(0))

exp.model is m                  # False — a copy
pg.eq(exp.model, m)             # True  — an equal one

exp.model.sym_rebind(units=128) # mutating the program...
m.units                         # 8 — ...leaves the search space intact

That independence is the point: two programs drawn from the same space must not alias, or tuning one trial would corrupt the others.

Only the parts that are topo=True get copied. The rule is per node, not per candidate — a value is copied exactly where it is held, i.e. where it has a tree position. Everything else passes through by reference:

Candidate shape	In the materialized program
a topo=True object	copied
a plain Python object	shared
a topo=False pg.Object	shared
a topo=True object inside a plain list	copied
a plain object inside a plain list	shared
a plain field inside a copied topo=True object	shared

A copied holder does not isolate what it holds

The last row is the one that bites. Copying a topo=True candidate says nothing about the opaque values inside it — every program that selects that candidate shares them:

class Tokenizer:            # a plain, non-symbolic object
    def __init__(self, name):
        self.name = name
        self.calls = 0

shared = Tokenizer('bpe')
space = Exp(tok=pg.oneof([shared, Tokenizer('wp')]))

a = pg.materialize(space, pg.DNA(0))
b = pg.materialize(space, pg.DNA(0))

a.tok is shared             # True — same object in every program
a.tok.calls += 1
b.tok.calls                 # 1 — state bleeds across trials

PyGX shares rather than copies here because it has no safe way to duplicate an arbitrary Python object — a model, a client, an open handle — and copying one silently would be worse than sharing it. If a trial mutates such a value, isolate it deliberately: declare it as a topo=True pg.Object so the tree owns it, or attach a pg.field(transform=copy.deepcopy).

Two habits follow. Don't reuse a value after passing it into a search space — it has been adopted, and belongs to the space now; pass pg.maybe_clone if you need to keep your own. And compare materialized values with pg.eq, never is.

Tying two fields to one decision

Sometimes two fields must move together: a decoder that mirrors its encoder's width, a stride that follows the kernel size. Writing the same pg.oneof expression twice does not do that — each call builds its own primitive, so they are two independent decisions and the tuner may pick differently for each:

class Model(pg.Object, topo=True):
    encoder_units: int
    decoder_units: int


# WRONG: two decisions, so the two can disagree.
space = Model(
    encoder_units=pg.oneof([64, 128, 256]),
    decoder_units=pg.oneof([64, 128, 256]),   # 9 combinations, not 3
)

Binding one primitive and reusing it does share a decision. A hyper primitive placed at several locations is one decision that lands everywhere:

class FlatModel(pg.Object):      # topo=False, the default
    encoder_units: int
    decoder_units: int

units = pg.oneof([64, 128, 256])
flat = FlatModel(encoder_units=units, decoder_units=units)

pg.dna_spec(flat)                       # ONE decision point
[(m.encoder_units, m.decoder_units) for m in pg.iter(flat)]
# [(64, 64), (128, 128), (256, 256)]

Under topo=True a node holds a single tree position, so placing the same object at two fields is refused (SharedHoldError). Spell the extra locations with pg.Ref, which shares the node — and, with it, the decision:

class Model(pg.Object, topo=True):
    encoder_units: int
    decoder_units: int

units = pg.oneof([64, 128, 256])
space = Model(encoder_units=units, decoder_units=pg.Ref(units))

pg.dna_spec(space).space_size           # 3, not 9

Sharing is by identity, not equality — two separately constructed pg.oneofs with the same candidates stay independent, which is why the first example in this section gives 9 combinations. Aliased fields each receive their own copy of the decided value, so a mutable choice is never aliased between them.

Giving two pg.oneofs the same name= does not unify them; it raises Found 2 decision point definitions clash on name 'units'.

When to reach for a reference instead

Reuse handles "these two fields are the same decision". A reference handles the cases reuse cannot:

• the follower must track a value that is not a hyper primitive (a plain field, or one decided elsewhere in the tree);
• you want to derive rather than mirror — see DerivedValue below.

pg.hyper.reference is the mechanism for this. It makes one field follow another instead of deciding for itself:

space = Model(
    encoder_units=pg.oneof([64, 128, 256]),
    decoder_units=pg.hyper.reference('encoder_units'),
)

for m in pg.iter(space):
    print(m.encoder_units, m.decoder_units)

# 64 64
# 128 128
# 256 256

The search space still has one decision point — a reference is not a choice, so it costs the tuner nothing:

pg.dna_spec(space)
# Space({
#   0 = 'encoder_units': Choices(num_choices=1, [
#     (0): 64
#     (1): 128
#     (2): 256
#   ])
# })

How the path resolves. The path is relative, searched from the reference's parent upward toward the root — so a sibling is named directly ('encoder_units'), and a value further up is found by the same walk. Dotted paths reach into other subtrees:

class Sub(pg.Object, topo=True):
    units: int = 0

class Top(pg.Object, topo=True):
    enc: Sub
    dec: Sub

space = Top(
    enc=Sub(units=pg.oneof([64, 128])),
    dec=Sub(units=pg.hyper.reference('enc.units')),
)
# enc.units == dec.units in every materialized program

Order does not matter: a reference may name a field declared after it.

What you get is a copy, by the same rule as any other materialized value (see above) — derive returns copy.copy of the referenced value, so a mutable target is not aliased between the two fields.

Deriving a value, not just mirroring one. To compute from one or more referenced values, subclass pg.hyper.DerivedValue and implement derive, which receives the resolved values positionally:

class Sum(pg.hyper.DerivedValue):

    def derive(self, *values):
        return sum(values)

    def custom_apply(self, path, value_spec, allow_partial,
                     child_transform=None):
        # Accept this marker wherever it is placed; see "Marker targets".
        return (False, self)


class M(pg.Object, topo=True):
    a: int = 0
    b: int = 0
    total: int = 0


space = M(a=pg.oneof([1, 2]), b=pg.oneof([10, 20]),
          total=Sum(reference_paths=['a', 'b']))

for m in pg.iter(space):
    print(m.a, m.b, m.total)

# 1 10 11
# 1 20 21
# 2 10 12
# 2 20 22

References need a tree to walk

A reference resolves through the symbolic tree, so its holder must be topo=True. On a topo=False object (the default since 0.4) there is no parent to search from, and materialization fails:

class Flat(pg.Object):        # topo=False
    a: int = 0
    b: int = 0

pg.materialize(Flat(a=pg.oneof([1, 2]), b=pg.hyper.reference('a')),
               pg.DNA(0))
# ValueError: Cannot resolve 'a': parent not found.

An unresolvable path fails the same way. And a reference may not point at another derived value — ValueError: Derived value (path=b) should not reference derived values. Point every reference at a real value.

Caveats

As shown above, for symbolic classes created with pg.symbolize, the __init__ method is delayed until the object becomes concrete. For symbolic classes created by subclassing pg.Object, the on_sym_bound, on_sym_post_init, and on_sym_change events are always triggered when the object is first created or later mutated, even when the object is abstract. For logic that requires a concrete self, override on_sym_ready instead — it fires only once the object is concrete, so every field is guaranteed present:

class MyObject(pg.Object):
    x: int
    y: int

    def on_sym_ready(self):
        super().on_sym_ready()
        # All symbolic attributes are guaranteed concrete here.
        self._z = self.x + self.y

If you must run logic in on_sym_bound itself (e.g. it has to fire even while the object is partial), guard the concrete-only parts with self.sym_abstract:

    def on_sym_bound(self):
        super().on_sym_bound()
        if not self.sym_abstract:
            self._z = self.x + self.y