Manifolds

hype.manifold

class hype.manifold.Manifold(*args, **kwargs)

Base class for all manifolds.

static dim(dim)

Add any additional dimensions necessary for the manifold

Parameters:dim (int) – dimension specified by user

Returns int

distance(u, v)

Compute the distance between u and v

Parameters:

• u (Tensor) – first tensor
• v (Tensor) – second tensor

Returns:

Distance between embeddings u and v

Return type:

Tensor

expm(p, d_p, lr=None, out=None)

Exponential map for manifold. Takes a point d_p in the tangent space of p and maps it on to the manifold

Parameters:

• p (Tensor) – reference point defining the tangent space
• d_p (Tensor) – point in p’s tangent space to be mapped on to the manifold

Returns:

d_p mapped on to the manifold

Return type:

Tensor

init_weights(w, scale=0.0001)

Initialize the weights of a Tensor

Parameters:

• w (Tensor) – Parameter to initialize
• scale (float) – Initialize uniformly in the range [-scale, scale]

Returns:

initialization is done in place

Return type:

None

logm(x, y)

Logarithmic map for manifold. Takes a point y located on the manifold and projects it into the tangent space of x

Parameters:

• x (Tensor) – reference point defining the tangent space
• y (Tensor) – point to be mapped into x’s tangent space

normalize(u)

Perform any type of normalization to a Tensor. Examples include fixing a vector to the Hyperboloid (lorentz model) or restricting the norm of a vector

Parameters:u (Tensor) – vectors to normalize Returns:Normalized tensor Return type:Tensor

ptransp(x, y, v, ix=None, out=None)

Parallel transport for manifold. Assuming v is in the tangent space of x, ptransp will perform parallel transport into the tangent space of y

Parameters:

• x (Tensor) – starting point
• y (Tensor) – end point
• v (Tensor) – point in tangent space

Returns:

embedding parallel transported from the tangent space of x to the tangent space of y

Return type:

Tensor

rgrad(p, d_p)

Given the euclidean gradient of p (d_p), computes the Riemannian gradient.

Parameters:

• p (Tensor) – embedding
• d_p (Tensor) – euclidean gradient of p

Returns:

Riemannian gradient of p

Return type:

Tensor

hype.lorentz

class hype.lorentz.LorentzManifold(eps=1e-12, _eps=1e-05, norm_clip=1, max_norm=1000000.0, debug=False, **kwargs)

Lorentz model of hyperbolic geometry. This is the manifold used in “Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry” (Nickel et al., 2018)

static dim(dim)

See dim()

distance(u, v)

See distance()

\(d(u, v) = \text{acosh}(-\langle u, v, \rangle_L)\)

expm(p, d_p, lr=None, out=None, normalize=False)

See expm()

\(exp_p(d_p) = \text{cosh}(||d_p||_L)p + \text{sinh}(||d_p||) \frac{d_p}{||d_p||_L}\)

init_weights(w, irange=1e-05)

Same as init_weights(), but also fixes the normalized embeddings to the hyperboloid

static ldot(u, v, keepdim=False)

Computes the Lorentzian Scalar Product between u and v

\(\langle u, v \rangle_L = -u_0 * v_0 + \sum_{i=1}^n u_i * v_i\)

Parameters:

• u (Tensor) – embedding
• v (Tensor) – embedding

Returns:

Tensor

logm(x, y)

See logm()

normalize(w)

See normalize()

ptransp(x, y, v, ix=None, out=None)

See ptransp()

rgrad(p, d_p)

See rgrad()

hype.poincare

class hype.poincare.PoincareManifold(eps=1e-05, **kwargs)

Poincaré Ball model of hyperbolic geometry. This is the manifold used in “Poincaré Embeddings for Learning Hierarchical Representations” (Nickel et al., 2017)

Parameters:eps (float) – \(\epsilon\) value to restrict the radius of the ball. This is used to prevent numerical overflow/underflow

static dim(dim)

Add any additional dimensions necessary for the manifold

Parameters:dim (int) – dimension specified by user

Returns int

distance(u, v)

See distance()

\(d(u, v) = \text{arcosh}\left( 1 + 2 \frac{||u - v||^2}{(1 - ||u||^2)(1 - ||v||^2)} \right)\)

expm(p, d_p, normalize=False, lr=None, out=None)

See expm()

init_weights(w, scale=0.0001)

Initialize the weights of a Tensor

Parameters:

• w (Tensor) – Parameter to initialize
• scale (float) – Initialize uniformly in the range [-scale, scale]

Returns:

initialization is done in place

Return type:

None

logm(p, d_p, out=None)

See logm()

normalize(u)

See normalize()

ptransp(p, x, y, v)

See ptransp()

rgrad(p, d_p)

See rgrad()

hype.euclidean

class hype.euclidean.EuclideanManifold(max_norm=1, **kwargs)

static dim(dim)

Add any additional dimensions necessary for the manifold

Parameters:dim (int) – dimension specified by user

Returns int

distance(u, v)

See distance()

\(d(u, v) = \sum_{i=0}^{n} (u_i - v_i)\)

expm(p, d_p, normalize=False, lr=None, out=None)

See expm()

init_weights(w, scale=0.0001)

Initialize the weights of a Tensor

Parameters:

• w (Tensor) – Parameter to initialize
• scale (float) – Initialize uniformly in the range [-scale, scale]

Returns:

initialization is done in place

Return type:

None

logm(p, d_p, out=None)

See logm()

normalize(u)

See normalize()

ptransp(p, x, y, v)

See ptransp()

rgrad(p, d_p)

See rgrad()

Dataloaders

hype.graph_dataset

class hype.graph_dataset.BatchedDataset

Create a dataset for training Hyperbolic embeddings. Rather than allocating many tensors for individual dataset items, we instead produce a single batch in each iteration. This allows us to do a single Tensor allocation for the entire batch and filling it out in place.

Parameters:

• idx (ndarray[ndims=2]) – Indexes of objects corresponding to co-occurrence. I.E. if idx[0, :] == [4, 19], then item 4 co-occurs with item 19
• weights (ndarray[ndims=1]) – Weights for each co-occurence. Corresponds to the number of times a pair co-occurred. (Equal length to idx)
• nnegs (int) – Number of negative samples to produce with each positive
• objects (list[str]) – Mapping from integer ID to hashtag string
• nnegs – Number of negatives to produce with each positive
• batch_size (int) – Size of each minibatch
• num_workers (int) – Number of threads to use to produce each batch
• burnin (bool) – perform frequency based negative sampling

next()

Get the next minibatch

Returns:inputs, targets Return type:Tuple[Tensor, Tensor]

nnegatives()

Number of negative samples to use.

Returns:int

Models

hype.graph

hype.graph.eval_reconstruction(adj, lt, distfn, workers=1, progress=False)

Reconstruction evaluation. Evaluate how well the embedding is able to reconstruct the original input graph. Specifically, for each node, we compute all of its nearest neighbors in the embedding space and rank them amongst its non-neighbors.

Parameters:

• adj (dict[int, set[int]]) – Adjacency list mapping objects to its neighbors
• lt (torch.Tensor[N, dim]) – Embedding table with N embeddings and dim dimensionality
• distfn (Callable[[Tensor, Tensor], Tensor]) – distance function to use for computing nearest neighbors in embedding space
• workers (int) – number of workers to use
• progress (bool) – display progress bar

Returns:

mean_rank, map_rank

Return type:

Tuple[float, float]

hype.graph.load_edge_list(path, symmetrize=False)

Load an edgelist dataset in CSV format. The CSV file must have at least 3 columns: id1, id2, and weight. If the dataset is directed, then it is assumed that id2 is the parent of id1.

Parameters:

• path (str) – path to the CSV file
• symmetrize (bool) – If set to True, then for every edge A -> B, we create a symmetric edge B -> A

Returns:

A tuple containiner: idx an array of edges, objects a list of the unique objects in the graph, and weights an array the same length of idx containing the weights of each edge

Return type:

Tuple[np.ndarray[ndim=2], list[str], np.ndarray[ndim=1]]

hype.sn

Optimizers

hype.rsgd

class hype.rsgd.RiemannianSGD(params, lr=<Mock name='mock.required' id='139711080874672'>, rgrad=<Mock name='mock.required' id='139711080874672'>, expm=<Mock name='mock.required' id='139711080874672'>)

Riemannian stochastic gradient descent.

Parameters:

• rgrad (Function) – Function to compute the Riemannian gradient from the Euclidean gradient
• retraction (Function) – Function to update the retraction of the Riemannian gradient

step(lr=None, **kwargs)

Performs a single optimization step.

Parameters:lr (float, optional) – learning rate for the current update.

Utils

hype.train

hype.train.train(device, model, data, optimizer, opt, log, rank=1, queue=None, ctrl=None, checkpointer=None, progress=False)

Function to train embeddings

Parameters:

• device (torch.device) – which device to train on
• model (torch.nn.Module) – model to train
• data (BatchedDataset or AdjacencyDataset) – dataloader
• optimizer (torch.optim.Optimizer) – optimizer
• opt (SimpleNamespace) – command line options
• log (logging.Logger) – log
• rank (int) – thread rank if using multiple training threads
• queue (multiprocessing.Queue) – Queue to put epoch stats into if using multiple threads/asynchronous control
• checkpointer (Callable) – checkpointing function
• progress (bool) – whether or not to display progress bar per epoch

hype.checkpoint

class hype.checkpoint.LocalCheckpoint(path, include_in_all=None, start_fresh=False)

Module for managing model checkpoints.

Parameters:

• path (str) – path to save the checkpoint to
• include_in_all (dict) – a dictionary of objects to save in every call to :func:save
• start_fresh (bool) – If True, then ignore any existing checkpoint,
• initialize from previous checkpoint (otherwise) –

initialize(params)

Initialize the checkpoint. If start_fresh is True, then params is returned. Otherwise if a checkpoint at self.path exists, the checkpoint is loaded and returned

Parameters:params (dict) – checkpoint contents Returns:Either params or the contents of the checkpoint stored at self.path Return type:dict

save(params, tries=10)

Save a checkpoint containing params merged with self.include_in_all

Parameters:

• params (dict) – data to store in checkpoint. This is merged with anything supplied to include_in_all in the constructor
• tries (int) – number of attempts to try and save the checkpoint. If the number of attempts exhausts, then no checkpoint is saved

Returns:

None

hype.common

class hype.common.Acosh

Inverse Hyperbolic Cosine

static forward(ctx, x, eps)

Inverse Hyperbolic Cosine

\(\text{acosh}(x) = \text{ln}(x + \sqrt{x^2 + 1})\)

Parameters:

• x (Tensor) – input
• eps (Float) – epsilon value to avoid NaN in backward pass