API reference¶

A thorough, type-annotated reference for the public API. For a one-screen scan of the whole surface, see the Cheat sheet; for the raw Doxygen-extracted signatures, see Autodoc.

Everything is in namespace tny (namespace cs = cuda::std). Include <teeny/teeny.h> for all of it; <teeny/cuda.h> adds the CUDA memory spaces.

Legend: Idx = the shape's index_type (int64_t for shape<...>). T = element type. A static index is an integral_constant (Int<k>()); a runtime index is a plain integer. "→ view" means a non-owning tensor<…, storage::view> aliasing the same memory (no copy).

The tensor type¶

template <class T, class Shape, class Layout = ccontiguous, storage O = storage::view>
struct tensor;

Slicing / permuting / peeling / .at() of a gpu tensor yields a storage::gpu_view (a non-owning view of device memory), so a device pointer is never mistaken for a host one in the type. Helpers: storage_is_device (gpu/gpu_view), storage_is_host_accessible, storage_is_view (view/gpu_view/pinned_view/mapped_view), storage_view_of(O) (the view kind that preserves a source's space). pinned/mapped are host-accessible, but a view of one keeps its space — pinned_view / mapped_view (so DLPack can label it kDLCUDAHost) — while behaving like a plain view otherwise.

Ownership aliases¶

Factories¶

Wrap existing memory (→ view):

Input → output type — wrap view facets. The element type T is deduced from the pointer (a const T* gives a read-only view); the extents come from the passed shape (each axis static/dynamic exactly as in its E); the ownership is always a view. Only the layout and the space vary with the call form (storage_view_of folds a named backend to its view kind, so wrap never yields an owner):

Allocate new storage — element type T defaults to float; static shape → stack (host+device), dynamic shape → heap (host only):

Geometry¶

Indexing & slicing¶

Axis args accept a runtime integer, a static Int<k>(), or a slice specifier. Negative integer indices wrap (count from the back).

Slice specifiers:

That vocabulary is closed: an index argument that is none of the above (nor an integer) is a compile error naming the mistake — notably a std::array/ std::tuple where the tuple-unpack form wants cuda::std's — never a silently kept full axis.

Assignment into a slice copies (broadcasts); on a named view it rebinds:

Input → output type — operator() per-axis arg. Element type is unchanged (const if the source is), the output layout is a folded strides<...>, and the space is storage_view_of(O). Each axis argument independently decides whether that axis survives and, if so, its output extent/stride staticity. The output rank = source rank − (number of integer args); ellipsis expands to rank − #other args copies of all before this table applies:

So t(i, all, slice<1,4>()) on <A,B,C> → <B,3>; t(i, all, slice(a,b)) on <A,B,C> → <B,-1>. A negative step (slice(none,none,-1)) reverses the axis (a negative folded stride); a strides<...> source stays fully sliceable.

Structure (views)¶

All return a view and work on any source layout (incl. strides<...>); axis template args are signed (negatives count from the back). Each <Ax> op also has a value form — pass Int<Ax>() in place of the template argument (t.squeeze(Int<1>()) == t.squeeze<1>(), likewise permute/flip/unsqueeze/ reshape/recast); the value spelling is the preferred one. The axis-list ops — permute, flip, squeeze, unsqueeze — additionally take an axis<...>{} tag (t.squeeze(axis<0,2>{}) == t.squeeze<0,2>(), t.flip(axis<0,2>{}) == t.flip<0,2>()), the same one peel/slice_along/ the reductions use. An empty list, axis<>{}, names no axis and is therefore a no-op for squeeze/unsqueeze/flip (numpy's axis=() rule) — distinct from the no-argument squeeze()/unsqueeze()/flip(), which keep their own meanings; permute accepts it only at rank 0. The reductions read it the same way — sum(a, axis<>{}) reduces over no axis and keeps a's shape, where the plain sum(a) (no axis argument at all) reduces over every axis, see Math & reductions. See Views & structure.

Type inference — what the output type keeps. A view op transforms the input type along four independent facets; staticity is preserved wherever it is derivable, so a kernel written against a static shape stays static through these ops (no accidental fallback to runtime extents/strides):

Worked input→output shapes (E = source extents):

reshape/flatten need exact C-contiguity (is_contiguous()); clone() first if the source isn't. recast does not — it only re-types the extents and keeps the source strides, so it works on any layout.

Input → output type — recast<NewE[, NewL]>(). No copy in any form; the element type T and the space are preserved (the result is storage_view_of(O)), the rank is unchanged, and every static dim of NewE recovers (folds into the type) a possibly-dynamic source dim — e.g. <-1,-1> → .recast<shape<-1,3,3>>() gives <-1,3,3>. The NewL argument chooses the strides + layout type:

An explicit NewL is a reinterpretation ("I promise it's laid out this way"); a debug build verifies the imposed/derived strides against the source's actual strides (symmetric with the static-extent check), UB only under -DNDEBUG.

Input → output type — to / clone. Whether the result is a borrow or an owning copy, and its ownership, per call:

The to<T2>()/clone() copying paths run copy_ on the HOST, so they cannot dereference DEVICE memory; the dynamic-shape (_TNY_HOST) overloads static_assert the source is host-accessible. To copy or convert a gpu/gpu_view tensor use the free to<Space>(t) (cuda.h) above, which is device-aware.

nd-peel (iteration)¶

Input → output type — peel cell. Each yielded cell is a view (storage_view_of(O) — gpu/gpu_view source → gpu_view), element type T unchanged, layout a folded strides<...>. The kept axes carry their source extents (static stays static) and folded strides; a peel_front<N> cell over <*batch(N), M, N> is <M, N> with the trailing extents and strides static even when the batch is dynamic. peel<Axes...> drops the named axes; the cell rank is source rank − #peeled axes.

Math¶

Element type of results follows promote(A,B) (C++ rules, but among floats the lower width wins — pytorch-style; -DTNY_STD_PROMOTION opts out). The result's offset index type is one that covers both operands' — the wider of the two for the usual (all-signed) pair, so a mixed int32/int64 broadcast → int64; a pair that disagrees in signedness steps up to a signed type wide enough for both ranges. Either way it is independent of the element promotion.

In-place (mutates *this, returns tensor&)¶

A tensor rhs broadcasts (numpy-style; needs equal rank — unsqueeze first); a scalar rhs applies to every element. An in-place op keeps *this's own offset index type (there is no new result to widen); the offset math runs internally in a type that covers every participant — both in width and in signedness — so a narrow-indexed destination is safe against a wide-indexed rhs, and an unsigned-indexed tensor is safe against a flipped (negative-stride) signed-indexed one — see Mixing widths.

Out-of-place (→ new tensor; static shape → stack, else heap)¶

into(dest) — write into a preallocated buffer. Pass into(y) as the last argument to any producer above to write the result into y (one fused pass, no allocation) and return y&, instead of a fresh tensor — the kernel-friendly out=. into(y) is a distinct type, so it never collides with a scalar argument (which is what lets add/sub also take the fused alpha). y may alias an operand and may have a different dtype — the arithmetic still runs in the operands' precision (scalar right-hand side and axpy coefficient included) and only the result is cast to y's type, so a.op(b, into(y)) produces exactly the numbers y.copy_(a.op(b)) would — including when the source is half/bfloat16: into(y) rounds through the twin's own promote_t (a 16-bit float there) before casting to y, the same extra rounding step the twin's temporary result goes through.

y may be a slice, written straight out of a view-producing op — cross(a, b, into(N(i, all))), sum(a, into(cells.at(i, j))), x.add(y, into(z.permute<1,0>())). Those ops return their view by value, and into() accepts such a temporary, so a slot of a bigger output needs no named intermediate; the write lands in the storage the slice refers to. Use the call for its effect and don't keep the dest& it returns (the temporary view is gone at the end of the statement). A temporary owning tensor is rejected at compile time (into(zeros<double>(shape<3>{}))) — its storage would die with the statement, so the result would be discarded.

y's shape is checked. A unary or scalar-rhs producer wants exact equality with the source (it has nothing to stretch); a tensor-rhs one applies its broadcast rule against the y you pass — each operand axis must equal y's or be 1. There is no stretching on the way out: only the operands broadcast, never the destination, so a y smaller in any axis is rejected, not written repeatedly. And because the tensor-rhs check pins the operands against y — not y against their natural result — y may deliberately be larger, or of higher rank, than the op alone would return: a.add(b, into(y)) is "y = a + b" with the allocation and copy saved, so the operands stretch to fill whatever broadcast-compatible y you supply, exactly as y.copy_(a + b) would. Every producer catches a mismatch the same way: a compile error whenever the extents in play are static, and a debug-time check (assert, off under -DNDEBUG) when any of them is dynamic.

auto a = zeros(shape<8,8>{});
auto y = zeros(shape<2,2>{});
a.mul(2.0, into(y));   // compile error: dest's shape must match the source's
exp(a, into(y));       // same
a.add(a, into(y));     // same, for the broadcasting tensor-rhs form

auto row = zeros(shape<1,3>{});
auto out = zeros(shape<2,3>{});
out.add(row, into(out));   // fine: the (1,3) OPERAND stretches over (2,3)
out.add(out, into(row));   // compile error: a (1,3) DEST does not stretch

auto big = zeros(shape<5,3>{});
row.add(row, into(big));   // fine, deliberate: row + row alone is (1,3), but big is
                           //   the result shape YOU chose — the operands stretch and
                           //   fill all five rows (== big.copy_(row + row))

y must not self-overlap either — no extent > 1 axis with stride 0 (only a hand-written wrap(ptr, e, strides-with-a-0) makes such a view). It would take many results into one element and keep just the last, so a debug-time check rejects it, for every into(dest) producer alike — the same guard that rejects an in-place write into one (see Math & broadcasting).

y may also carry a different offset index type from the operands' (a narrower one included) — the widening needed to walk both safely happens inside the engine, in width and in signedness alike, and y keeps its own type; see Mixing widths.

Comparisons → a bool tensor (broadcast), reduced with .all()/.any()¶

Reductions¶

Accumulate in the reduce type — double for small floats (float/double/half; a wider float keeps its own type), the item type for integers — then cast the result back to the tensor's element type T (sum(float_tensor) → float, computed in double). A leading type argument makes that type both the accumulator and the result.

Every reduction here (plus sqnorm/norm/sqdist/dist below, and allclose) is also a method — a.sum(), a.mean(), a.dot(b), a.sqdist(b), a.allclose(b), a.sum<0>(), a.mean(axis<1>{}), a.norm(), a.sum(into(cell)) — with the same overload shapes as the free form. The free sum(a) spelling stays (generic code and the numpy reader both use it).

Axis reductions: a fully static result → stack (host+device); any dynamic result → heap (host only). keepdims applies to sum/prod/max/min/mean/sqnorm/norm. Axis lists must be distinct but may be given in any order — with or without keepdims (sum<2,0>(a, keepdims) == sum<0,2>(a, keepdims)), like every other axis-list op. A repeated axis is a compile error in every spelling — sum<0,0>(a), sum(a, axis<0,0>{}), a.sum<0,0>(), mean<1,-2>(a) on a rank-3 tensor (negatives are normalised before the check) — not a silently dropped duplicate. The explicit <Acc, Axes...> template split stays (C++17 has no universal template parameter to unify "leading type = accumulator" vs "leading int = axis"), but past that split every TRAILING keyword — dtype<Acc>{}, axis<...>{}, keepdims, into(dest) — composes generically, in any subset and any order (tny::_kw in kwargs.h): sum(a, dtype<double>{}, axis<0>{}, keepdims, into(dest)) == sum<double, 0>(a, keepdims, into(dest)). dot composes the same way (dtype and into; it has no axis concept).

Reductions also take into(dest) (the out= spelling). A full reduction writes its scalar into a rank-0 destination — sum(a, into(cell)), dot(a, b, into(cell)), where cell is a local<T, shape<>>{} or a rank-0 view over an address wrap(&x, shape<>{}) (a non-rank-0 dest is a static_assert, so forgetting the <axes> fails to compile instead of splatting the total). An axis reduction copies its lower-rank result into dest — sum<0>(m, into(buf)), and the axis<...> value form takes into too (mean(m, axis<1>{}, into(buf))); its dest is broadcast-compatible with the reduced shape (it goes through copy_). Dtype casts — for every pair of element types, half into bfloat16 and back included — and the destination is returned by reference.

Vector algebra & geometry¶

Contained exact linear-algebra / geometry on views (no solves, inversion, or optimisation — those stay out of teeny).

Input → output type — result element type. The accumulator and the result dtype are separate. Default accumulator = reduce_type_t<T> (double for a small float float/double/half/bfloat16; a wider float keeps itself; int64_t/uint64_t for a narrow integer, so sum/prod/dot can't overflow mid-accumulation; an already-≥8-byte integer keeps itself). A leading type arg makes that type both the accumulator and the result.

The axis forms follow the same element-type rule; only the shape changes (named axes removed) and the ownership splits static → stack / dynamic → heap.

Half precision¶

Dispatch (the anyrank boundary)¶

Input → output type — fixed-rank views. The rank-erased carrier hands out a dyn_tensor<T, offset_t, R, Space> = tensor<T, cs::dextents<offset_t, R>, cs::layout_stride, Space> — a fully dynamic, arbitrarily-strided view whose Space is the carrier's view space (storage::view by default; storage::gpu_view for a device carrier):

Recover static extents with .recast<NewE>() at the kernel boundary (see the recast type mapping) to fold the inner shape back into the type.

DLPack strides are in elements; numpy __array_interface__ in bytes (divide by the itemsize first).

Compile flags¶