Hermes Vector

TechSponsored

Sponsored content from Hutton Tech Solutions.

Hutton Tech Solutions Offers Local Private AI Inference

Hutton Tech Solutions provides private AI inference infrastructure in Kamloops, allowing businesses to process sensitive data locally without sending it to cloud providers.

By Sponsored by Hutton Tech Solutions

Also in: Español · Português · Français · Deutsch · हिन्दी · 中文

Checked against 1 sourceai.huttontech.solutions
Illustration for: Hutton Tech Solutions Offers Local Private AI Inference

In brief

  • The company says private AI inference runs on dedicated hardware in Kamloops, ensuring data never leaves Canada or crosses international borders.
  • According to Hutton Tech Solutions, the infrastructure is HIPAA, SOC2, and PIPEDA compliant, addressing regulatory concerns for healthcare, legal, and financial sectors.
  • The company states that the service uses an OpenAI-compatible API, allowing existing software to connect by changing the endpoint URL.

Hutton Tech Solutions Offers Local Private AI Inference

Hutton Tech Solutions has introduced a service focused on private AI inference, designed specifically for businesses in Kamloops that require strict data sovereignty and regulatory compliance. The company says that traditional cloud AI services, such as those from OpenAI or Claude, involve sending sensitive information to servers in foreign jurisdictions, which may be subject to laws like the CLOUD Act and used for model training. In contrast, Hutton Tech Solutions offers a local alternative where data remains under the control of the business.

What the Service Provides

Private AI inference, as described by the company, involves running AI models on dedicated hardware that the client controls. This infrastructure is physically located in Kamloops, ensuring that data never leaves the building or crosses international borders. The company states that this approach eliminates logging, meaning no queries or responses are stored, and prevents data from being used to train other models. This setup is presented as a solution for industries where data privacy is critical, including healthcare, legal, financial services, and government sectors.

The technology relies on enterprise-grade NVIDIA DGX systems. According to the company, these systems feature 192GB of unified memory, Grace Blackwell architecture GPUs, and high-speed networking with sub-millisecond latency. The infrastructure is designed to support large models, such as Llama 3 with up to 405B parameters, without the rate limits often associated with cloud services. The company notes that this local setup can offer faster response times for simple queries, typically between 50-200ms, compared to cloud services that may have higher base latency due to network travel.

Who It Is For

The company identifies several key sectors that benefit from this local infrastructure. For healthcare providers, Hutton Tech Solutions says the service allows for the analysis of patient records and generation of clinical summaries without violating HIPAA or provincial health acts. In the legal sector, the service is positioned to help firms maintain attorney-client privilege by avoiding the risks associated with sending privileged communications to third-party cloud services. Financial institutions and government entities are also cited as primary users, as they are bound by strict data protection regulations that prohibit sending client information to third-party clouds.

The company provides a case study of a Kamloops medical clinic that implemented this solution. According to Hutton Tech Solutions, the clinic used a dedicated NVIDIA Blackwell GPU with a fine-tuned Llama 3 70B model to assist with clinical documentation. The company states that this resulted in a 90% reduction in documentation time and $15,000 in monthly savings compared to hiring medical scribes, while maintaining 100% HIPAA compliance.

Pricing and Implementation

Hutton Tech Solutions offers a monthly subscription model for dedicated infrastructure. The company states that monthly subscriptions start at $2,000 per month, with no long-term contracts required. This cost includes unlimited usage, no rate limits, and full compliance support. The company compares this to cloud costs, noting that for businesses processing three million or more tokens per day, private AI can be more cost-effective than cloud services like OpenAI GPT-4, which the company estimates would cost $10,800 annually for similar volume.

Implementation typically takes two to four weeks. The process begins with an assessment of use cases and compliance needs, followed by the deployment of hardware and configuration of VPN security. The company says the API is OpenAI-compatible, meaning existing applications can connect to the private infrastructure by simply changing the endpoint URL. Hutton Tech Solutions handles all model updates, maintenance, and optimization, ensuring clients have access to the latest AI capabilities without managing the underlying technology.

Hutton Tech Solutions is a private company and is not listed on any stock exchange.


Sources

Sponsored content from Hutton Tech Solutions. It was written with AI assistance from Hutton Tech Solutions' own materials and checked against them; claims are the company's. Cover image is AI-generated.

Questions readers ask

What is Hutton Tech Solutions' private AI inference service?
Hutton Tech Solutions offers a local private AI inference service for businesses in Kamloops that keeps data under the client's control on dedicated hardware. This infrastructure ensures data never leaves the building or crosses international borders, eliminating logging and preventing use for model training.
What hardware does Hutton Tech Solutions use for its private AI service?
The technology relies on enterprise-grade NVIDIA DGX systems featuring 192GB of unified memory and Grace Blackwell architecture GPUs. These systems provide high-speed networking with sub-millisecond latency to support large models like Llama 3 with up to 405B parameters.
Who is the target audience for this service?
The service is designed for industries requiring strict data privacy, including healthcare, legal, financial services, and government sectors. It helps these entities comply with regulations such as HIPAA, provincial health acts, and attorney-client privilege requirements.
How much does the service cost?
Monthly subscriptions start at $2,000 per month with no long-term contracts required. This cost includes unlimited usage, no rate limits, and full compliance support.
What are the implementation details and timeline?
Implementation typically takes two to four weeks, beginning with an assessment of use cases and compliance needs. The process involves deploying hardware, configuring VPN security, and connecting via an OpenAI-compatible API.

The day's checked news, by email

One short email a day with the top stories. Free, and you can unsubscribe in one click.

Or follow for the latest news:

How do you feel about this story?

Comments 0

No comments yet. Start the discussion.