Incidents | DeviceCloud Incidents reported on status page for DeviceCloud https://status.devicecloud.dev/ https://d1lppblt9t2x15.cloudfront.net/logos/7fe4a2bfb496420d003efc17ad1666cd.png Incidents | DeviceCloud https://status.devicecloud.dev/ en Android delays https://status.devicecloud.dev/incident/783085 Tue, 09 Dec 2025 16:34:00 -0000 https://status.devicecloud.dev/incident/783085#55bcd8bf4f74786b989242ec58b34f89f6a2defa66f4d1f44faeb72db94780f9 The blockage appears to be resolved. We're actively monitoring. Android delays https://status.devicecloud.dev/incident/783085 Tue, 09 Dec 2025 16:15:00 -0000 https://status.devicecloud.dev/incident/783085#a440de04ceba07cec797056ad987305b8804c088741f8798a84cb6725447d691 We're investigating a hold up in our Android pipeline Android Test Delays https://status.devicecloud.dev/incident/782202 Mon, 08 Dec 2025 13:38:00 -0000 https://status.devicecloud.dev/incident/782202#5af91d52619676fdc35758ec16d42fa17098eeb7e8784ce887e9734e11fcb592 The issue has been resolved. Android Test Delays https://status.devicecloud.dev/incident/782202 Mon, 08 Dec 2025 10:51:00 -0000 https://status.devicecloud.dev/incident/782202#de2cc162085450a7df2c20865aaaec4bc971828eb8c48e7d1e9f5905eb21125c We're investigating a failed caching component that is cause test delays. Docs recovered https://status.devicecloud.dev/ Fri, 05 Dec 2025 09:13:16 +0000 https://status.devicecloud.dev/#ac4f54b658e26ced24530f7f56b3489f94a89d851dd12825fb2e6f26e9654ce1 Docs recovered Docs went down https://status.devicecloud.dev/ Fri, 05 Dec 2025 08:48:58 +0000 https://status.devicecloud.dev/#ac4f54b658e26ced24530f7f56b3489f94a89d851dd12825fb2e6f26e9654ce1 Docs went down Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:57:00 -0000 https://status.devicecloud.dev/incident/779588#8206a14750fa03a4f0b3fc734f26d38e32a937539be08d5f08f9a0a8ce793511 Network issue resolved Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:57:00 -0000 https://status.devicecloud.dev/incident/779588#8206a14750fa03a4f0b3fc734f26d38e32a937539be08d5f08f9a0a8ce793511 Network issue resolved Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:57:00 -0000 https://status.devicecloud.dev/incident/779588#8206a14750fa03a4f0b3fc734f26d38e32a937539be08d5f08f9a0a8ce793511 Network issue resolved Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:54:00 -0000 https://status.devicecloud.dev/incident/779588#6c85fc84bd8d8abe36392075d7a82c9e2838bf7163bc6ce7a487ee993433f8cd Informed by our data centre supplier of a network issue. Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:54:00 -0000 https://status.devicecloud.dev/incident/779588#6c85fc84bd8d8abe36392075d7a82c9e2838bf7163bc6ce7a487ee993433f8cd Informed by our data centre supplier of a network issue. Network issues at our Wales data centre https://status.devicecloud.dev/incident/779588 Thu, 04 Dec 2025 11:54:00 -0000 https://status.devicecloud.dev/incident/779588#6c85fc84bd8d8abe36392075d7a82c9e2838bf7163bc6ce7a487ee993433f8cd Informed by our data centre supplier of a network issue. Degraded Performance https://status.devicecloud.dev/incident/778882 Wed, 03 Dec 2025 12:10:00 -0000 https://status.devicecloud.dev/incident/778882#57143e64d1e7af04dceb7ccb7ef0def3e9487dc5231ff9f4666d820504e6360e Blockage resolved. Degraded Performance https://status.devicecloud.dev/incident/778882 Wed, 03 Dec 2025 11:57:00 -0000 https://status.devicecloud.dev/incident/778882#742ed1b5cd382c3f87f8e35b268eb477e298dcbe0ca59e3050e8865af2701878 We're currently investigating backup on our Android pipeline. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:59:00 -0000 https://status.devicecloud.dev/incident/775384#b1ce88d80c9724d7ce8e6216a54231aadf829675fd95d29b8761d1419db359c8 Power has come back online and jobs are resuming. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:59:00 -0000 https://status.devicecloud.dev/incident/775384#b1ce88d80c9724d7ce8e6216a54231aadf829675fd95d29b8761d1419db359c8 Power has come back online and jobs are resuming. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:59:00 -0000 https://status.devicecloud.dev/incident/775384#b1ce88d80c9724d7ce8e6216a54231aadf829675fd95d29b8761d1419db359c8 Power has come back online and jobs are resuming. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:41:00 -0000 https://status.devicecloud.dev/incident/775384#fe3821551be5317e36fda01d507a5eb111143aeef583d6096ab3d017772132b2 Our main data center in Wales has been affected by a power failure which is causing outages. We're investigating. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:41:00 -0000 https://status.devicecloud.dev/incident/775384#fe3821551be5317e36fda01d507a5eb111143aeef583d6096ab3d017772132b2 Our main data center in Wales has been affected by a power failure which is causing outages. We're investigating. Power failure https://status.devicecloud.dev/incident/775384 Thu, 27 Nov 2025 17:41:00 -0000 https://status.devicecloud.dev/incident/775384#fe3821551be5317e36fda01d507a5eb111143aeef583d6096ab3d017772132b2 Our main data center in Wales has been affected by a power failure which is causing outages. We're investigating. Android Delays https://status.devicecloud.dev/incident/775182 Thu, 27 Nov 2025 11:30:00 -0000 https://status.devicecloud.dev/incident/775182#4d3c227245b44774d73c185c77a0ec78d9b0696ee8acb255265dbfc28343c0e8 An internal networking component failed and delays occurred whilst the system recovered. Android Delays https://status.devicecloud.dev/incident/775182 Thu, 27 Nov 2025 10:15:00 -0000 https://status.devicecloud.dev/incident/775182#87db327c4eeeabb5c8456642f81eaae3f34dc374f3315677f207f48a826050ef Some jobs stuck due to errored storage component. We're investigating. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 19:20:00 -0000 https://status.devicecloud.dev/incident/774085#1c8e1c02c9be2761a1d01ecade4029c19dae6662a26b32026ac12db022f2948b We traced the issue to a caching layer related to last week's Supabase downtime. Jobs are now progressing through the system. Please raise a support request if your job still appears stuck. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 19:20:00 -0000 https://status.devicecloud.dev/incident/774085#1c8e1c02c9be2761a1d01ecade4029c19dae6662a26b32026ac12db022f2948b We traced the issue to a caching layer related to last week's Supabase downtime. Jobs are now progressing through the system. Please raise a support request if your job still appears stuck. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 19:20:00 -0000 https://status.devicecloud.dev/incident/774085#1c8e1c02c9be2761a1d01ecade4029c19dae6662a26b32026ac12db022f2948b We traced the issue to a caching layer related to last week's Supabase downtime. Jobs are now progressing through the system. Please raise a support request if your job still appears stuck. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 18:01:00 -0000 https://status.devicecloud.dev/incident/774085#d11f0b59accb3509a6eeb4d1937e39e5dd5463eb56e7daa47df52fc1511167e7 We're seeing delays across test runners. Investigating root cause. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 18:01:00 -0000 https://status.devicecloud.dev/incident/774085#d11f0b59accb3509a6eeb4d1937e39e5dd5463eb56e7daa47df52fc1511167e7 We're seeing delays across test runners. Investigating root cause. Test Delays https://status.devicecloud.dev/incident/774085 Tue, 25 Nov 2025 18:01:00 -0000 https://status.devicecloud.dev/incident/774085#d11f0b59accb3509a6eeb4d1937e39e5dd5463eb56e7daa47df52fc1511167e7 We're seeing delays across test runners. Investigating root cause. Authentication Issues https://status.devicecloud.dev/incident/773089 Mon, 24 Nov 2025 11:03:00 -0000 https://status.devicecloud.dev/incident/773089#297f806822306f5bd8fbfe923dfaf9dca614718803cab9300b046e207dde9a20 Authentication error rates appear to have returned to normal. Authentication Issues https://status.devicecloud.dev/incident/773089 Mon, 24 Nov 2025 10:21:00 -0000 https://status.devicecloud.dev/incident/773089#371d4e22f186b4acca8f2028bd85583e0f5d75073d830dea1e889ad278aeb163 We're seeing increased failure rate on authentication requests since 9:32 AM. Appears to be related to Cloudflare timeout. Investigation underway. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:59:00 -0000 https://status.devicecloud.dev/incident/770816#49d08d4ccef7bcec4154343c895ae782652c987acfa0c7ffff0f5b71c9b7d5c9 We've identified the issue and released a patch. Job have been re-submitted and are running normally again. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:59:00 -0000 https://status.devicecloud.dev/incident/770816#49d08d4ccef7bcec4154343c895ae782652c987acfa0c7ffff0f5b71c9b7d5c9 We've identified the issue and released a patch. Job have been re-submitted and are running normally again. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:59:00 -0000 https://status.devicecloud.dev/incident/770816#49d08d4ccef7bcec4154343c895ae782652c987acfa0c7ffff0f5b71c9b7d5c9 We've identified the issue and released a patch. Job have been re-submitted and are running normally again. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:38:00 -0000 https://status.devicecloud.dev/incident/770816#b5f5cfd8319c575c461d189a9f20911982ffe517b3841c413b0a65097bc9e81c We have some users seeing this error which is related to yesterday's emergency storage switchover. We're actively investigating. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:38:00 -0000 https://status.devicecloud.dev/incident/770816#b5f5cfd8319c575c461d189a9f20911982ffe517b3841c413b0a65097bc9e81c We have some users seeing this error which is related to yesterday's emergency storage switchover. We're actively investigating. "Failed to download app" error https://status.devicecloud.dev/incident/770816 Thu, 20 Nov 2025 08:38:00 -0000 https://status.devicecloud.dev/incident/770816#b5f5cfd8319c575c461d189a9f20911982ffe517b3841c413b0a65097bc9e81c We have some users seeing this error which is related to yesterday's emergency storage switchover. We're actively investigating. Binary upload issue https://status.devicecloud.dev/incident/770118 Wed, 19 Nov 2025 18:37:00 -0000 https://status.devicecloud.dev/incident/770118#c86585de419eb02fca7ae306cfa765b23e491a126d7fe10f96613ee9ff2282ce We're still seeing issues with our main storage provider and have switched to our cold backup via CLI version 4.1.7 Binary upload issue https://status.devicecloud.dev/incident/770118 Wed, 19 Nov 2025 10:26:00 -0000 https://status.devicecloud.dev/incident/770118#a070deff731d6cbc39a54e5907cae06558f786509ee79cd3cdac2380de265d18 Upstream Supabase storage is encountering an issue meaning some users cannot upload new binaries. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 18:03:00 -0000 https://status.devicecloud.dev/incident/756832#37faa2906e454ab72487b7840fe3b03d54ca810930fca556bf6697a6ed5e931a Network issue is now resolved and job are progressing through the system. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 18:03:00 -0000 https://status.devicecloud.dev/incident/756832#37faa2906e454ab72487b7840fe3b03d54ca810930fca556bf6697a6ed5e931a Network issue is now resolved and job are progressing through the system. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 18:03:00 -0000 https://status.devicecloud.dev/incident/756832#37faa2906e454ab72487b7840fe3b03d54ca810930fca556bf6697a6ed5e931a Network issue is now resolved and job are progressing through the system. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 17:25:00 -0000 https://status.devicecloud.dev/incident/756832#5bbcd1f3803f91f9bf6e17ac59ef7160dd4801b9d389f16f2c0a8048715109be We've applied a fix and jobs are starting to come back online. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 17:25:00 -0000 https://status.devicecloud.dev/incident/756832#5bbcd1f3803f91f9bf6e17ac59ef7160dd4801b9d389f16f2c0a8048715109be We've applied a fix and jobs are starting to come back online. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 17:25:00 -0000 https://status.devicecloud.dev/incident/756832#5bbcd1f3803f91f9bf6e17ac59ef7160dd4801b9d389f16f2c0a8048715109be We've applied a fix and jobs are starting to come back online. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 12:32:00 -0000 https://status.devicecloud.dev/incident/756832#2d95e2ca94a2040174436dd7677f1829de09a72e8674c305c0f8f6c32de87700 Traced to firmware issue - we're deploying redundancy systems. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 12:32:00 -0000 https://status.devicecloud.dev/incident/756832#2d95e2ca94a2040174436dd7677f1829de09a72e8674c305c0f8f6c32de87700 Traced to firmware issue - we're deploying redundancy systems. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 12:32:00 -0000 https://status.devicecloud.dev/incident/756832#2d95e2ca94a2040174436dd7677f1829de09a72e8674c305c0f8f6c32de87700 Traced to firmware issue - we're deploying redundancy systems. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 11:03:00 -0000 https://status.devicecloud.dev/incident/756832#edb4b46f7b4693d8909ff542f37d8c6064af42f3752787ef321d5834aa248030 We're actively investigating a network issue. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 11:03:00 -0000 https://status.devicecloud.dev/incident/756832#edb4b46f7b4693d8909ff542f37d8c6064af42f3752787ef321d5834aa248030 We're actively investigating a network issue. Network Issues Affecting iOS and Android Runners https://status.devicecloud.dev/incident/756832 Mon, 03 Nov 2025 11:03:00 -0000 https://status.devicecloud.dev/incident/756832#edb4b46f7b4693d8909ff542f37d8c6064af42f3752787ef321d5834aa248030 We're actively investigating a network issue. Login and result emails not being delivered https://status.devicecloud.dev/incident/747089 Mon, 20 Oct 2025 10:15:00 -0000 https://status.devicecloud.dev/incident/747089#d2cb083496ad39154fcf921e389fbe3dbaa1f22491bda8616ce9801b44afcdb6 Emails are working again Login and result emails not being delivered https://status.devicecloud.dev/incident/747089 Mon, 20 Oct 2025 10:15:00 -0000 https://status.devicecloud.dev/incident/747089#d2cb083496ad39154fcf921e389fbe3dbaa1f22491bda8616ce9801b44afcdb6 Emails are working again Login and result emails not being delivered https://status.devicecloud.dev/incident/747089 Mon, 20 Oct 2025 09:22:00 -0000 https://status.devicecloud.dev/incident/747089#9d4dd305e28d5ede6cea864081173e26cb68f944150a4aa35851c7d6f8c15299 Caused by upstream provider: https://resend-status.com Login and result emails not being delivered https://status.devicecloud.dev/incident/747089 Mon, 20 Oct 2025 09:22:00 -0000 https://status.devicecloud.dev/incident/747089#9d4dd305e28d5ede6cea864081173e26cb68f944150a4aa35851c7d6f8c15299 Caused by upstream provider: https://resend-status.com API recovered https://status.devicecloud.dev/ Thu, 16 Oct 2025 20:52:39 +0000 https://status.devicecloud.dev/#c2fed71d6817aa321124f760ca3c426f2a3dc3c5ab9ac1e313d2a0dfeda9d5be API recovered API went down https://status.devicecloud.dev/ Thu, 16 Oct 2025 20:50:40 +0000 https://status.devicecloud.dev/#c2fed71d6817aa321124f760ca3c426f2a3dc3c5ab9ac1e313d2a0dfeda9d5be API went down Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:18:00 -0000 https://status.devicecloud.dev/incident/706009#8ad41079d2d50f650f3370921ad8d874fc6211be4bbd26b45caa3c4cf3d6927f Android jobs now returning to normal. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:18:00 -0000 https://status.devicecloud.dev/incident/706009#8ad41079d2d50f650f3370921ad8d874fc6211be4bbd26b45caa3c4cf3d6927f Android jobs now returning to normal. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:18:00 -0000 https://status.devicecloud.dev/incident/706009#8ad41079d2d50f650f3370921ad8d874fc6211be4bbd26b45caa3c4cf3d6927f Android jobs now returning to normal. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:16:00 -0000 https://status.devicecloud.dev/incident/706009#12cd97386f7d4722f7adec97eaf895e46e02ba70de956729724e8cdbd3c2d41e iOS jobs are now returning to normal Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:16:00 -0000 https://status.devicecloud.dev/incident/706009#12cd97386f7d4722f7adec97eaf895e46e02ba70de956729724e8cdbd3c2d41e iOS jobs are now returning to normal Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 12:16:00 -0000 https://status.devicecloud.dev/incident/706009#12cd97386f7d4722f7adec97eaf895e46e02ba70de956729724e8cdbd3c2d41e iOS jobs are now returning to normal Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:53:00 -0000 https://status.devicecloud.dev/incident/706009#09b2b0ac157167b25e290c4b6c084f248f4c098a0452c99e80bb94ea6dad1cff We're currently investigating some network downtime. Apologies for the issues. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:53:00 -0000 https://status.devicecloud.dev/incident/706009#09b2b0ac157167b25e290c4b6c084f248f4c098a0452c99e80bb94ea6dad1cff We're currently investigating some network downtime. Apologies for the issues. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:53:00 -0000 https://status.devicecloud.dev/incident/706009#09b2b0ac157167b25e290c4b6c084f248f4c098a0452c99e80bb94ea6dad1cff We're currently investigating some network downtime. Apologies for the issues. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:20:00 -0000 https://status.devicecloud.dev/incident/706009#22eba5256c83e7b8cc3de9c72d8acb23a5d71772ecca5e9743cf48bf82444731 Currently investigating a network issue which is causing delays in tests. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:20:00 -0000 https://status.devicecloud.dev/incident/706009#22eba5256c83e7b8cc3de9c72d8acb23a5d71772ecca5e9743cf48bf82444731 Currently investigating a network issue which is causing delays in tests. Test Run Delays https://status.devicecloud.dev/incident/706009 Thu, 14 Aug 2025 11:20:00 -0000 https://status.devicecloud.dev/incident/706009#22eba5256c83e7b8cc3de9c72d8acb23a5d71772ecca5e9743cf48bf82444731 Currently investigating a network issue which is causing delays in tests. iOS Runners Degraded Service https://status.devicecloud.dev/incident/701263 Wed, 06 Aug 2025 09:03:00 -0000 https://status.devicecloud.dev/incident/701263#45bbb90b87d0086c3b1af3ea788a732f8c9f17b7ddc33f9c595f44e1613dda13 Issue traced, patch created and now seeing iOS runners returning to normal. iOS Runners Degraded Service https://status.devicecloud.dev/incident/701263 Wed, 06 Aug 2025 09:03:00 -0000 https://status.devicecloud.dev/incident/701263#45bbb90b87d0086c3b1af3ea788a732f8c9f17b7ddc33f9c595f44e1613dda13 Issue traced, patch created and now seeing iOS runners returning to normal. iOS Runners Degraded Service https://status.devicecloud.dev/incident/701263 Wed, 06 Aug 2025 08:11:00 -0000 https://status.devicecloud.dev/incident/701263#243a986df6c54ee3f107034f8f8984f9c0833b371f750b590a67184389e2ae29 Some users experiencing "TypeError: Child process has already terminated." errors in their iOS runners. We're actively investigating. iOS Runners Degraded Service https://status.devicecloud.dev/incident/701263 Wed, 06 Aug 2025 08:11:00 -0000 https://status.devicecloud.dev/incident/701263#243a986df6c54ee3f107034f8f8984f9c0833b371f750b590a67184389e2ae29 Some users experiencing "TypeError: Child process has already terminated." errors in their iOS runners. We're actively investigating. API failing to correctly resolve Android versions for some devices. https://status.devicecloud.dev/incident/653015 Wed, 30 Jul 2025 05:20:00 -0000 https://status.devicecloud.dev/incident/653015#48d8e76e467d9b30fe5035d3d89029b52a253c719fae28eb5a7c5297fc907733 API is now operating correctly again. Users may need to re-submit affected Android jobs. API failing to correctly resolve Android versions for some devices. https://status.devicecloud.dev/incident/653015 Wed, 30 Jul 2025 03:10:00 -0000 https://status.devicecloud.dev/incident/653015#e96c9ba35e6d300034923b73a46d5611528eeab3d1533d3a94b2453778f442c5 API regression identified where some device to API level combinations of Android devices were incorrectly rejected by the device compatibility endpoint.