Visualcare Worker Mobile App (vWorker) Outage 05 October 2025

05/10/25 2:18pm

Visualcare Worker Mobile App (vWorker)

3hrs 58min

Customers were unable to connect to or synchronise data with the Visualcare Worker Mobile App during the outage period. All inbound traffic to the Mobile API was unavailable until service restoration.

Between , the Visualcare Mobile API experienced a full-service interruption. During this period, users of the Visualcare Worker Mobile App were unable to connect or synchronise due to a fault in one of the mobile infrastructure’s internal routing components.

The disruption followed an within Visualcare’s , which affected the connection between internal systems responsible for routing encrypted traffic. Because all communication inside Visualcare’s platform enforces strict encryption and validation, affected traffic was safely rejected by policy rather than being exposed or processed insecurely.

The issue was resolved through the deployment of a built on Visualcare’s current hardened infrastructure standard. The replacement restored normal mobile API traffic while maintaining the same high-security posture and encryption enforcement policies.

An automated infrastructure update applied to a legacy routing component within Visualcare’s introduced a mismatch in how secure internal connections were validated.

As part of Visualcare’s defence-in-depth architecture,

Following the update, internal traffic between the legacy load balancer and the primary API tier was incorrectly classified as non-secure. In accordance with Visualcare’s security enforcement policies, these connections were automatically rejected, resulting in a complete interruption of inbound Mobile API traffic.

The affected component resided within a legacy configuration pending decommission and did not include the modern validation logic applied to current load balancer standards.

The graph below shows the network activity for the primary Mobile API load balancer.

The combined disk latency and I/O graphs show a correlated spike at the time of the automated update. This behaviour reflects short-lived internal retries as the legacy load balancer attempted to establish new sessions with the primary API infrastructure after its connections were rejected. Because all traffic operated within the private cloud and remained encrypted, these retries generated transient increases in read/write activity and latency.

Performance stabilised immediately following the deployment of the new ALB, returning both latency and I/O to normal baseline levels.

Disk Latency and IO Spike from automated update and high rejection rate

Automated system update applied; primary Mobile API endpoint stops accepting secure traffic.

First customer issue reported.

P0 incident declared; Engineering initiated Incident Response Plan (IRP).

Primary Mobile API server identified as fault source; services and host restarted without success.

Automated update confirmed as root cause; rollback attempted.

Rollback completed but issue persisted; replacement of ALB authorised.

New ALB built, tested, and promoted to production.

Full traffic restoration confirmed; incident declared resolved.

The affected legacy load balancing stack has been fully decommissioned and replaced with the standardised, security-validated ALB configuration driven by our automated infrastructure provisioning.

A comprehensive review of all Application Load Balancer configurations has been completed across environments to ensure alignment with Visualcare’s current security and routing standards.

New monitoring rules have been implemented to detect and alert unusual patterns of internal connection rejections within our private cloud infrastructure. These rules specifically track attempted connections between the load balancer and primary API tier that fail security validation, enabling earlier detection of similar anomalies.

The list below shows the total outage time and uptime for each primary service in 2025.