“I forgot that every little action of the common day makes or unmakes character,” Oscar Wilde wrote in De Profundis

went into a rabbit hole into nvidia software.

• nvidia nim: apis to run llm models
• nvidia NeMo
  • curator: scalable data pre processing and curation toolkit for LLMs
  • customizer
  • evaluator
  • retriever
  • guardrails: toolkit for easily adding programmable guardrails to LLM-based conversational systems.
  • framework
• TensorRT-LLM: python api to define llms and build TensorRT engines that contain state-of-the-optimizes inference efficiently on NVIDIA GPUs.
• RAPIDS: GPU accelerated data science/analytics via open source software libraries and APIs.
  • cuGraph: accelerated graph analytics

also did a bit of reading into graph neural networks.

main takeaway is you can create a graph structure from text and images, but it's more useful for heterogeneous structures where the number of neighbors for each node in a graph is variable (as apposed to fixed for text and images).

there are three main predictive tasks for GNNs

(1) graph-level

• ex: molecule as a graph, predict its smell or probability of binding to a receptor
• analogy for image: classify an entire image
• analogy for text: label sentiment of an entire sentence

(2) node-level

• ex: predict identity/role of each node
• analogy for image: image segmentation
• analogy for text: parts-of-speech

(3) edge-level

• ex: image scene understanding, given nodes that represent objects in an image, predict which of these nodes share an edge or what the value of the edge is

the challenges of graphs in ML: representing graphs for neural networks

graphs have 4 types of info

1. nodes
2. edges
3. global-context
4. connectivity (hard part)

first three is straightforward, we create a node matrix N, where each node has an index i that stores the feature for node_i

connectivity is more complicated. first, adjacency matrix are sparse and space inefficient. second, many adjacency matrices can encode the same information, but without a guarantee that they produce the same result (they are not permutation invariant)

solution: represent sparse matrices as adjacency list

they describe connectivity of edge e_k between nodes n_i and n_j as a tuple (i, j) in the kth entry of the list.

conceptual

• A Gentle Introduction to Graph Neural Networks
• Understanding Convolutions on Graphs

applied

• CS224W: Machine Learning with Graphs
• Colab Notebooks and Video Tutorials — pytorch_geometric documentation
• Introduction to Graph Neural Networks with NVIDIA cuGraph-DGL | NVIDIA Technical Blog

found these datasets to work with

• Open Graph Benchmark | A collection of benchmark datasets, data-loaders and evaluators for graph machine learning in PyTorch.
• Yelp Dataset