Weavatrix Hetero and Hostwatch: private distributed AI across Mac and Linux
A working Mac–Linux model split shows what resource sharing means in practice: an owner controls each contribution, Hostwatch coordinates the workers, and Hetero places the model. These notes examine the integration run recorded on 7 October and reviewed for publication on 10 October.
What we tested: one model, two different machines
The useful starting point for distributed AI is a concrete model and a concrete ownership boundary. In the recorded Hostwatch integration, a Mac offered six GiB of Metal memory while a small Linux container offered 3.2 GiB and two CPU threads. Weavatrix Hetero placed Qwen3-4B across those contributions. The Mac ran the first twenty-six layers and the Linux worker ran the remaining ten. Neither worker's layer allowance was large enough for all thirty-six layers. Together, the measured offers supported a complete placement without asking either owner to lend the machine's entire memory.
The answer was compared with the whole model running on the same Mac using the same generation settings. Its recorded hash matched, and the answer was byte-identical. This is a useful correctness check for that prompt, model and configuration. The example demonstrates an actual private compute network and a working heterogeneous split. It also gives an operator a repeatable way to review a deployment: record the model identity, numerical settings, prompt, placement, memory offers and output comparison. We use the integration date for these field notes; the publication timestamp records the later review.
Hostwatch coordinates access; Hetero coordinates the model
Hostwatch supplies the account-scoped network, machine enrollment, owner settings, heartbeat directory, job queue and small coordination artifacts. Weavatrix Hetero supplies the worker runtime and the model coordinator. The coordinator probes workers, measures their offers and decides which layers belong on which device. This separation keeps the two products understandable. The infrastructure dashboard answers who owns a worker, whether it is online and what it is allowed to contribute. The model runtime answers whether a proposed layout fits, which stage should run where and how a generation request moves through the stages.
For a full distributed generation, Hetero uses the network's scoped credential and its coordinator workflow. The hub is a rendezvous and governance point. Stage execution and the large data path use the worker connections established for the run. This matters when sizing Hostwatch: forwarding a few queue messages has a different cost from carrying model weights or every hidden-state tensor through the web application. Keep the coordinator, stage listeners and model source visible in deployment notes. A successful hub login proves access to the network; the stage connection and completed generation prove the actual execution path.
Connect a worker through an explicit private invitation
A machine joins a selected private network using a one-use invitation. The worker exchanges that invitation for its own credential, stores the credential in a local token file and begins reporting its capabilities. The invitation has an expiry and cannot be reused after enrollment. This is an intentional act by the machine's owner or administrator. Creating a Hostwatch account does not grant access to the servers operating the Hostwatch service. The network directory is scoped to the authenticated account, and a worker credential belongs to its enrolled machine rather than becoming a general credential for the whole infrastructure.
A practical connection flow starts in the worker connection guide: choose the private network, create the invitation and run the generated worker command on the machine you control. Review the reported operating system, architecture, core count, available backends and memory. Then adjust its contribution before submitting meaningful work. Use a separate invitation for each additional machine. If an administrator removes a worker, its old credential should cease to authorize it. Clear enrollment instructions and distinct credential roles make the setup useful for a freelancer with one computer as well as for a company with several servers.
A memory budget is an allowance, not the machine inventory
The worker reports what hardware exists; the owner decides what may be lent. Hostwatch resolves CPU threads from the selected power share and records per-backend memory allowances. A machine can support CPU, Metal or another GPU backend while lending only a subset of those resources. In the integration run, the Linux worker's eighty-percent setting resolved to two threads and 3.2 GiB. The Mac's six-GiB Metal contribution was also an explicit allowance. These values belong in the dashboard beside current health and work state, so an operator can distinguish total installed memory from the budget actually available to a model.
On Apple silicon, CPU and Metal use the same underlying unified memory. Treat their displayed allowances as contributions subject to the same physical capacity, rather than adding them into an imaginary independent pool. Hetero's placement includes model ends, context-dependent key/value storage and per-stage requirements. The relevant question is whether a proposed execution layout fits its measured budgets with enough headroom for the host's other work. Begin conservatively, observe resident memory and latency during a run, and increase the share only when the machine's normal responsibilities still have their required resources. The control guide explains the owner-facing settings.
Read the placement numbers before starting the model
The recorded Qwen3-4B run used a context of 2,048 positions. After accounting for model ends and cache requirements, the Mac offered 5.15 GiB for layers and the Linux worker offered 2.70 GiB. The selected stages required approximately 5.09 and 2.04 GiB respectively. Those are the values that explain why this layout worked. They are more useful than adding six and 3.2 and calling the sum a model limit. The run also recorded different layer timings for Metal and CPU. Hetero used the offers to choose a feasible boundary and report its estimated compute time per verification pass.
A second configuration included a larger coordinator draft alongside the model ends. That configuration was screened before workers launched because its remaining layer capacity did not cover the complete model. The successful measured configuration used an n-gram draft and placed all thirty-six layers. This preflight check is part of controlled resource sharing: first resolve the owner's allowance, then reserve the execution overhead, then verify a complete layout. It gives the operator an actionable answer before starting a costly run. Save the accepted plan with the generation result so a later comparison can explain a different stage split or memory footprint.
Keep jobs accountable from submission to completion
Hostwatch's compute queue records a job kind, idempotency key, deadline, attempt and execution state. A worker leases an eligible job, renews its lease while working and returns a result associated with that attempt. Lease tokens prevent an obsolete worker attempt from overwriting a newer attempt's result. An idempotency key makes a retried submission return the existing job rather than quietly launching duplicate work. These details are especially useful when a laptop sleeps, a worker restarts or the coordinator reconnects. The dashboard can explain what happened through explicit states instead of presenting one indefinite loading indicator.
The queue also has bounds. The current hub allows a finite number of queued jobs and limits the size and total quota of coordination artifacts. Small inputs and results can use the hub artifact path; large model transfers belong in the configured model-distribution path. A private network credential authorizes only that network's work. In normal operation, review both the job state and the worker heartbeat. A completed queue message, a listening stage and a final generated answer are distinct milestones. Keeping them separate makes a resource-sharing interface useful as an operational tool rather than a decorative list of connected devices.
Make the directory describe usable capacity
The owner can pause lending, select accepted job categories, limit simultaneous jobs and set working hours. Workers report whether they are currently willing to accept work. The directory now combines that status with heartbeat freshness, the pause setting, eligible job kinds and occupied slots when reporting availability through MCP. An online worker with all slots occupied is visible as connected but unavailable for another immediate assignment. An expired heartbeat is unavailable even if the last remembered status said it was willing. This is a small operational distinction with a large effect on the clarity of a fleet view.
We verified the directory behavior with regression cases covering an expired heartbeat, a paused machine, a full worker, a worker declining work and a machine accepting no job kinds. We also checked that a separate account sees none of the first account's machines. The model coordinator can still use a queue for work that is expected to wait; the directory's immediate availability tells the operator what is usable now. A clear UI should show configured allowance, fresh status and active work together. That combination makes it possible to adjust a budget while understanding the effect on both model jobs and the rest of the machine.
Use MCP for bounded probes and research jobs
The Hostwatch MCP connection exposes an operation catalog and governed execution. For compute, a client can inspect its networks, list jobs, submit an allowed model probe or token-continuation job, retrieve the bounded result and cancel a job it owns. A submission requires the write scope, an owner role and explicit confirmation in the operation request. The submitted model name refers to an installed local model directory. Input limits bound prompt token arrays, new-token counts and artifact size. A small model probe is a useful first check because it verifies routing and worker capability without asking the client to orchestrate a complete model server.
Full stage placement and distributed serving remain the Hetero coordinator's workflow. Connect an MCP client to inspect or run the supported bounded tasks, then use Hetero's CLI for a complete distributed generation. Keep the network key in its intended local credential file and retain the job identifier in your run notes. The CLI guide and compute overview describe these paths from the user's perspective. This arrangement lets an agent request useful compute through an accountable interface while preserving the distinction between a constrained job and the authority required to establish a multi-stage serving topology.
Choose workers according to the data they will process
Account isolation and resource limits establish who may submit work and how much a worker may lend. The owner of a participating machine still controls that machine's operating system. Choose trusted private workers for confidential prompts, model checkpoints or research inputs. Review the stage listener's binding address, allowed peers and transport path before moving sensitive data. A private network is a governance boundary; the execution environment is another boundary that deserves its own review. Keeping both visible is more useful than describing distributed inference as an abstract cloud in which every participating computer is automatically trusted.
Community contribution is a separate, explicit opt-in setting. Decide which workloads are appropriate for contributed workers and keep private work scoped to its intended network. Use independent credentials for administrators, coordinators and machines. Revoke a departed worker and rotate an exposed credential. For company deployments, connect only machines the administrator is authorized to enroll, then document who can change budgets and submit jobs. The private compute security guide provides the setup checklist. The result is a comprehensible workflow: an owner grants a bounded contribution, an authorized coordinator schedules it, and the worker handles only the work permitted by its settings.
A repeatable first run for your own network
Begin with two trusted machines and one small installed model. Enroll each machine separately, set conservative CPU and memory shares, and confirm fresh heartbeats. Run a bounded probe through the intended interface. Inspect the resulting offers and select a context size that fits the complete placement. Execute a short generation using a fixed prompt and deterministic settings. Record the model revision, stage ranges, backend, context, allowances, actual memory observations and output. Then run the same configuration on a single machine that can hold the complete model, and compare the answer with the distributed result.
Once correctness is established for that case, measure the workload you actually need: a larger context, a representative prompt set, concurrent requests or a different network path. Compare completion time and host pressure alongside output quality. Keep a pause control available to the machine owner and preserve the accepted placement with each result. The Hostwatch integration shows a practical route from two separate computers to one coordinated AI workload. Its value is concrete: permissions and budgets stay visible, the model has a complete measured placement, and the owner can trace a submitted job to the machine work and answer it produced.
Start with a private network, explicit per-machine budgets and a small verified job. Let Hetero place the model using measured offers, keep the control plane separate from the data path, and compare the distributed answer with a single-machine reference.