Visualization#

Various aspects of the contraction process can be visualized using the following functions.

%config InlineBackend.figure_formats = ['svg']
import cotengra as ctg

Hypergraph visualization#

The first visualization we can do is to visualize the hypergraph corresponding to the geometry of the tensor network or equation with HyperGraph.plot. Hyper edges are shown as a zero size vertex connecting the tensors they appear on, like a COPY-tensor:

inputs, output, shapes, size_dict = ctg.utils.rand_equation(
    6,
    2,
    n_out=1,
    n_hyper_in=1,
    n_hyper_out=1,
    seed=4,
)
print(ctg.utils.inputs_output_to_eq(inputs, output))
ctg.get_hypergraph(inputs, output, size_dict).plot();
abce,bdf,bdef,a,b,ab->ca
_images/042c968308679079ed3c62e53ea53f78ee70d5510090ce5aa4430e9d322c9b98.svg

This simple contraction has five types of index:

  1. standard inner index - ‘e’ - which appear on exactly two tensors

  2. standard inner multi-indices - ‘d’, ‘f’ - which both appear on the same two tensors

  3. standard outer index - ‘c’ - which appears on exactly one tensor and the output

  4. hyper inner index - ‘b’ - which appears on more than two tensors

  5. hyper outer index - ‘a’ - which appears on multiple tensors and the output

The nodes and indices are assigned unique colors by default, with hyper indices shown as dashed lines.

Small tree visualization#

If the network is small enough and we have a ContractionTree for it, we can also visualize its entirety including all indices involved at each intermediate contraction using the ContractionTree.plot_flat method:

tree = ctg.array_contract_tree(inputs, output, size_dict)
tree.plot_flat();
_images/17f97c44318dfd5e16f97f0331ff248aa44c6f52ec8a53856de30c5c69e6e3ad.svg

Here the unique node and index coloring by default matches that of the default hypergraph visualization. The contraction flows from bottom to top.

For the remaining examples, we’ll generate a larger 2D lattice contraction, with no outputs or hyper indices:

# generate an equation representing a 2D lattice
inputs, output, shapes, size_dict = ctg.utils.lattice_equation([5, 6])
hg = ctg.get_hypergraph(inputs, output, size_dict)
hg.plot(draw_edge_labels=True);
_images/f30dbe66831422a931105317ce433b46b4cb8c55b24d31ef14962193daa9fce0.svg

You can turn off the node/edge coloring or set the node coloring to a simple centrality measure like so:

hg.plot(node_color="centrality", edge_color=False);
_images/16e9f60f5326496035e4fcdbf6490315de47a8dfa6adb068b826800676da2a90.svg

Optimizer trials visualization#

If we run an hyper optimizer, we can visualize how the scores progress with trials using the HyperOptimizer.plot_trials method:

opt = ctg.HyperOptimizer(methods=['greedy', 'kahypar', 'labels'], progbar=True)

# run it and generate a tree
tree = opt.search(inputs, output, size_dict)
log2[SIZE]: 6.00 log10[FLOPs]: 3.46: 100%|██████████| 128/128 [00:02<00:00, 51.89it/s]

By default the y-axis is the objective score, but you can specify e.g. 'flops'’:

opt.plot_trials('flops');
_images/bc8cb4db3f4ca861b4b585c8592852a635287b097fb070c282934d9d1e5a2050.svg

We can also plot the distribution of contraction costs against contraction widths using the HyperOptimizer.plot_scatter method:

opt.plot_scatter(x='size', y='flops');
_images/06f703501ce015d3d78612428590f3c0b1f556da3d7b19b04985f646a5679dea.svg

Large Tree visualizations#

The following visualization functions are available for inspecting a single, complete ContractionTree once generated. They mostly wrap plot_tree, where you can see most of the extra options.

Contractions#

tree.plot_contractions gives you an overview of the memory and costs throughout the contraction:

tree.plot_contractions();
_images/c1e240f41c6c4db0a3eadd30cde045d7540898d5ccd37994297421d91fd9d717.svg

Here, peak is the memory required for all intermediates to be stored at once, write is the size of the new intermedite tensor, the max of which is the contraction width. cost is the scalar operations of each contraction.

The list of corresponding pairwise contractions can be explicitly shown with the print_contractions method:

tree.print_contractions()
(0) cost: 1.6e+01 widths: 3.0,2.0->3.0 type: tensordot
inputs: ih[j],k[j]->
output: ih(k)

(1) cost: 3.2e+01 widths: 3.0,3.0->4.0 type: tensordot
inputs: ih[k],[k]vu->
output: ih(vu)

(2) cost: 6.4e+01 widths: 4.0,4.0->4.0 type: tensordot
inputs: [i]hv[u],[i]ts[u]->
output: hv(ts)

(3) cost: 6.4e+01 widths: 4.0,3.0->5.0 type: tensordot
inputs: h[v]ts,[v]GF->
output: hts(GF)

(4) cost: 1.3e+02 widths: 5.0,4.0->5.0 type: tensordot
inputs: h[t]sG[F],[t]ED[F]->
output: hsG(ED)

(5) cost: 1.6e+01 widths: 2.0,3.0->3.0 type: tensordot
inputs: R[W],PV[W]->
output: R(PV)

(6) cost: 3.2e+01 widths: 3.0,3.0->4.0 type: tensordot
inputs: [R]PV,G[R]Q->
output: PV(GQ)

(7) cost: 6.4e+01 widths: 4.0,4.0->4.0 type: tensordot
inputs: [P]VG[Q],E[P]O[Q]->
output: VG(EO)

(8) cost: 1.3e+02 widths: 5.0,4.0->5.0 type: tensordot
inputs: hs[GE]D,V[GE]O->
output: hsD(VO)

(9) cost: 6.4e+01 widths: 4.0,3.0->5.0 type: tensordot
inputs: C[N]MO,[N]UV->
output: CMO(UV)

(10) cost: 2.6e+02 widths: 5.0,5.0->6.0 type: tensordot
inputs: hsD[VO],CM[O]U[V]->
output: hsD(CMU)

(11) cost: 2.6e+02 widths: 6.0,4.0->6.0 type: tensordot
inputs: hs[DC]MU,r[C]B[D]->
output: hsMU(rB)

(12) cost: 6.4e+01 widths: 3.0,4.0->5.0 type: tensordot
inputs: [g]fh,[g]rqs->
output: fh(rqs)

(13) cost: 2.6e+02 widths: 6.0,5.0->5.0 type: tensordot
inputs: [hs]MU[r]B,f[hr]q[s]->
output: MUB(fq)

(14) cost: 1.6e+01 widths: 2.0,3.0->3.0 type: tensordot
inputs: [a]b,[a]lm->
output: b(lm)

(15) cost: 3.2e+01 widths: 3.0,3.0->4.0 type: tensordot
inputs: [b]lm,c[b]d->
output: lm(cd)

(16) cost: 6.4e+01 widths: 4.0,4.0->4.0 type: tensordot
inputs: l[mc]d,[c]n[m]o->
output: ld(no)

(17) cost: 6.4e+01 widths: 4.0,3.0->5.0 type: tensordot
inputs: l[d]no,e[d]f->
output: lno(ef)

(18) cost: 1.3e+02 widths: 5.0,4.0->5.0 type: tensordot
inputs: ln[oe]f,[e]p[o]q->
output: lnf(pq)

(19) cost: 2.6e+02 widths: 5.0,5.0->6.0 type: tensordot
inputs: MUB[fq],ln[f]p[q]->
output: MUB(lnp)

(20) cost: 2.6e+02 widths: 6.0,4.0->6.0 type: tensordot
inputs: MU[B]ln[p],[p]Az[B]->
output: MUln(Az)

(21) cost: 6.4e+01 widths: 4.0,3.0->5.0 type: tensordot
inputs: A[L]KM,[L]TU->
output: AKM(TU)

(22) cost: 2.6e+02 widths: 6.0,5.0->5.0 type: tensordot
inputs: [MU]ln[A]z,[A]K[M]T[U]->
output: lnz(KT)

(23) cost: 1.3e+02 widths: 5.0,4.0->5.0 type: tensordot
inputs: l[nz]KT,[n]yx[z]->
output: lKT(yx)

(24) cost: 6.4e+01 widths: 5.0,3.0->4.0 type: tensordot
inputs: [l]KTy[x],[l]w[x]->
output: KTy(w)

(25) cost: 6.4e+01 widths: 4.0,4.0->4.0 type: tensordot
inputs: [K]T[y]w,[y]JI[K]->
output: Tw(JI)

(26) cost: 3.2e+01 widths: 4.0,3.0->3.0 type: tensordot
inputs: [T]w[J]I,[J]S[T]->
output: wI(S)

(27) cost: 1.6e+01 widths: 3.0,3.0->2.0 type: tensordot
inputs: [wI]S,[w]H[I]->
output: S(H)

(28) cost: 4.0e+00 widths: 2.0,2.0->0.0 type: tensordot
inputs: [SH],[HS]->
output: 


The indices are colored according to:

  1. blue - appears on left input and is kept

  2. green - appears on right input and is kept

  3. red - appears on both inputs and is contracted away

  4. purple - appears on both inputs and is kept (a ‘batch’ or hyper index)

Tent#

The most general purpose visualization for the structure of a ContractionTree is the ContractionTree.plot_tent method. This plots the input network (in grey) at the bottom, and the contraction tree intermediates laid out above. The width and color of the tree edges denote the intermediate tensor widths, and the size and color of the tree nodes denote the FLOPs required to contract each intermediate tensor:

tree.plot_tent();
_images/92d27d6f2b56fb3cb8f0fd06024a65db33dfeab038b65f02459cde0c3f79d5a2.svg

If you supply order=True then the intermediate nodes will be in the exact vertical order than they would be performed:

tree.plot_tent(order=True);
_images/cd8fe1085dc22dd876371ac681734e0a46bdfbf3d98931415c780280b2c6f583.svg

Note

If you have sliced indices, these will appear as dashed lines in the input graph.

Ring#

Another option is the ContractionTree.plot_ring method which lays out the input network on a ring, with the contraction tree intermediates laid out towards the center. The more arcs cross between branches the more expensive that contraction. This can be useful for inspecting how many ‘spines’ a contraction has or how balanced it is:

tree.plot_ring();
_images/b744a5640e2c528d0e8cf8e68d1522116d8eb9a87a99c0ea501d35a78f5fccda.svg

Rubberband#

For small and close to planar graphs, an alternative visualization is the ContractionTree.plot_rubberband method. method using quimb. Here, nodes of the input graph are hierarchically grouped into bands according to the contraction tree. The order of contraction is represented by the colors:

tree.plot_rubberband();
_images/b32c4a0cd1d27a17583f36da299e2306726d8bbb0b4406df52c45676b98bb892.svg

All of the above methods can be pretty extensively customized, including by supplying custom colormaps. They also return (fig, ax) for further customization or embedding in other plots.

inputs, output, shapes, size_dict = ctg.utils.lattice_equation([5, 5, 5])
opt = ctg.HyperOptimizer(progbar=True, reconf_opts={}, minimize='combo-256')
tree = opt.search(inputs, output, size_dict)
log2[SIZE]: 31.00 log10[FLOPs]: 11.54: 100%|██████████| 128/128 [00:17<00:00,  7.23it/s]
fig, ax = tree.plot_tent(
    raw_edge_alpha=0.0,
    edge_colormap='Greens',
    node_colormap='Greens',
    edge_scale=2,
    node_scale=0.5,
    colorbars=False,
    tree_root_height=-1.0,
    figsize=(10, 10),
)
_images/30825e9ad2c4b78e3466d2d9866305436c26874690f6feb02f8719db69ba37ce.svg