GOOD
good
This incident has been resolved.
Jul 17, 6:13 PM UTC
minor
We are investigating reports of degraded performance for Codespaces
Jul 17, 5:56 PM UTC
good
This incident has been resolved.
Jul 17, 5:06 PM UTC
minor
Git Operations is operating normally.
Jul 17, 5:06 PM UTC
minor
Pages is operating normally.
Jul 17, 5:06 PM UTC
minor
Packages is operating normally.
Jul 17, 5:06 PM UTC
minor
We're continuing to investigate reports of issues with multiple services. We will continue to keep users updated on progress.
Jul 17, 4:47 PM UTC
minor
Packages is experiencing degraded performance. We are continuing to investigate.
Jul 17, 4:47 PM UTC
minor
Issues is operating normally.
Jul 17, 4:46 PM UTC
minor
Pages is experiencing degraded performance. We are continuing to investigate.
Jul 17, 4:30 PM UTC
minor
Issues is experiencing degraded performance. We are continuing to investigate.
Jul 17, 4:28 PM UTC
minor
We are investigating reports of issues with service(s): Git, Actions, Rules and Permissions, SSH authentication, and Authorization.. We will continue to keep users updated on progress towards mitigation.
Jul 17, 4:24 PM UTC
minor
We are investigating reports of degraded performance for Git Operations
Jul 17, 4:21 PM UTC
good
This incident has been resolved.
Jul 16, 3:07 AM UTC
major
Copilot chat is error rates are improving and we are continuing to monitor system health.
Jul 16, 2:56 AM UTC
major
Copilot chat is experiencing elevated error rates. We have identified the root cause and are working on remediation.
Jul 16, 2:10 AM UTC
major
Copilot chat is experiencing elevated error rates and we are currently investigating the issue.
Jul 16, 1:35 AM UTC
major
We are currently investigating this issue.
Jul 16, 12:53 AM UTC
good
This incident has been resolved.
Jul 13, 7:27 PM UTC
major
Copilot is operating normally.
Jul 13, 7:26 PM UTC
major
Our upstream provider continues to recover and we expect services to return to normal as more progress is made. We will provide another update by 20:00 UTC.
Jul 13, 6:01 PM UTC
major
Our upstream provider is making good progress recovering and we are validating that services are nearing normal operations. We will provide another update by 18:00 UTC.
Jul 13, 4:09 PM UTC
major
Our upstream provider is gradually recovering the service. We will provide another update at 23:00 UTC.
Jul 13, 11:18 AM UTC
major
We are continuing to wait on our upstream provider to see full recovery. We will provide another update at 11:00 UTC
Jul 13, 3:50 AM UTC
major
The error rate for Copilot chat requests remains steady at less than 10%. We are continuing to investigate with our upstream provider.
Jul 13, 3:20 AM UTC
major
Copilot is experiencing degraded performance. We are continuing to investigate.
Jul 13, 2:20 AM UTC
major
We have applied several mitigations to Copilot chat, reducing errors to less than 10% of all chat requests. We are continuing to investigate the issue with our upstream provider.
Jul 13, 2:19 AM UTC
major
Copilot chat is experiencing degraded performance, impacting up to 60% of all chat requests. We are continuing to investigate the issue with our upstream provider.
Jul 13, 1:32 AM UTC
major
Copilot chat is currently experiencing degraded performance, impacting up to 60% of all chat requests. We are investigating the issue.
Jul 13, 12:49 AM UTC
major
Copilot is experiencing degraded availability. We are continuing to investigate.
Jul 13, 12:29 AM UTC
major
Copilot API chat experiencing significant failures to backend services
Jul 13, 12:18 AM UTC
major
We are investigating reports of degraded performance for Copilot
Jul 13, 12:18 AM UTC
good
On July 11, 2024, between 10:20 UTC and 14:00 UTC Copilot Chat was degraded and experienced intermittent timeouts. This only impacted requests routed to one of our service region providers. The error rate peaked at 10% for all requests and 9% of users. This was due to host upgrades in an upstream service provider. While this was a planned event, processes and tooling was not in place to anticipate and mitigate this downtime.


We are working to improve our processes and tooling for future planned events and escalation paths with our upstream providers.

Jul 11, 3:21 PM UTC
minor
Copilot is operating normally.
Jul 11, 3:21 PM UTC
minor
We have mitigated the intermittent timeout errors impacting Copilot’s Chat functionality and expect the incident to be resolved shortly.
Jul 11, 3:19 PM UTC
minor
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. The timeout errors we are seeing has reduced back to healthy levels for the last 60 minutes but we are monitoring closely.
Jul 11, 3:04 PM UTC
minor
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.
Jul 11, 2:14 PM UTC
minor
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.
Jul 11, 1:32 PM UTC
minor
Copilot's Chat functionality is experiencing intermittent timeouts, we are investigating the issue.
Jul 11, 1:02 PM UTC
minor
We are investigating reports of degraded performance for Copilot
Jul 11, 1:02 PM UTC
good
On July 8th, 2024, between 18:18 UTC and 19:11 UTC, various services relying on static assets were degraded, including user uploaded content on github.com, access to docs.github.com and Pages sites, and downloads of Release assets and Packages.

The outage primarily affected users in the vicinity of New York City, USA, due to a local CDN disruption.

Service was restored without our intervention.

We are working to improve our external monitoring, which failed to detect the issue and will be evaluating a backup mechanism to keep critical services available, such as being able to load assets on GitHub.com, in the event of an outage with our CDN.

Jul 08, 7:45 PM UTC
minor
Issues and Pages are operating normally.
Jul 08, 7:44 PM UTC
minor
Issues and Pages are experiencing degraded performance. We are continuing to investigate.
Jul 08, 7:44 PM UTC
minor
Issues is operating normally.
Jul 08, 7:44 PM UTC
minor
Issues is experiencing degraded performance. We are continuing to investigate.
Jul 08, 7:44 PM UTC
minor
Pages and Issues are operating normally.
Jul 08, 7:44 PM UTC
minor
Our assets are serving normally again and all impact is resolved.
Jul 08, 7:44 PM UTC
minor
We are beginning to see recovery of our assets and are monitoring for additional impact.
Jul 08, 7:16 PM UTC
minor
githubstatus.com may not be available or may be degraded for some users in some regions.
Jul 08, 7:12 PM UTC
minor
We are investigating issues with loading assets, including JavaScript assets, on various parts of the site for some users.
Jul 08, 7:02 PM UTC
minor
We are investigating reports of degraded performance for Issues and Pages
Jul 08, 7:01 PM UTC
good
On July 5, 2024, between 16:31 UTC and 18:08 UTC, the Webhooks service was degraded, with customer impact of delays to all webhook delivery. On average, delivery delays were 24 minutes, with a maximum of 71 minutes. This was caused by a configuration change to the Webhooks service, which led to unauthenticated requests sent to the background job cluster. The configuration error was repaired and re-deploying the service solved the issue. However, this created a thundering herd effect which overloaded the background job queue cluster which put its API layer at max capacity, resulting in timeouts for other job clients, which presented as increased latency for API calls.

Shortly after resolving the authentication misconfiguration, we had a separate issue in the background job processing service where health probes were failing, leading to reduced capacity in the background job API layer which magnified the effects of the thundering herd. From 18:21 UTC to 21:14 UTC, Actions runs on PRs experienced approximately 2 minutes delay and maximum of 12 minutes delay. A deployment of the background job processing service remediated the issue.

To reduce our time to detection, we have streamlined our dashboards and added alerting for this specific runtime behavior. Additionally, we are working to reduce the blast radius of background job incidents through better workload isolation.

Jul 05, 8:57 PM UTC
major
We are seeing recovery in Actions start times and are observing for any further impact.
Jul 05, 8:44 PM UTC
major
We are still seeing about 5% of Actions runs taking longer than 5 minutes to start. We are scaling and shifting resources to encourage recovery of the problem.
Jul 05, 8:32 PM UTC
major
We are still seeing about 5% of Actions runs taking longer than 5 minutes to start. We are evaluating mitigations to increase capacity to decrease latency.
Jul 05, 7:58 PM UTC
major
We are seeing about 5% of Actions runs not starting within 5 minutes. We are continuing investigation.
Jul 05, 7:19 PM UTC
major
We have seen recovery of Actions run delays. Keeping the incident open to monitor for full recovery.
Jul 05, 6:40 PM UTC
major
Webhooks is operating normally.
Jul 05, 6:10 PM UTC
major
We are seeing delays in Actions runs due to the recovery with webhook deliveries. We expect this to resolve with the recovery of webhooks.
Jul 05, 6:09 PM UTC
major
Actions is experiencing degraded performance. We are continuing to investigate.
Jul 05, 6:07 PM UTC
major
We are seeing recovery as webhooks are being delivered again. We are burning down our queue of events. No events have been lost. New webhook deliveries will be delayed while this process recovers.
Jul 05, 5:57 PM UTC
major
Webhooks is experiencing degraded performance. We are continuing to investigate.
Jul 05, 5:55 PM UTC
major
We are reverting a configuration change that is suspected to contribute to the problem with webhook deliveries.
Jul 05, 5:42 PM UTC
major
Our telemetry shows that most webhooks are failing to be delivered. We are queueing all undelivered webhooks and are working to remediate the problem.
Jul 05, 5:20 PM UTC
major
Webhooks is experiencing degraded availability. We are continuing to investigate.
Jul 05, 5:17 PM UTC
major
We are investigating reports of degraded performance for Webhooks
Jul 05, 5:04 PM UTC
good
On July 3, 2024, between 1:34 PM UTC and 4:42 PM UTC the GitHub documentation was degraded and showed a 500 on non-cached pages. On average, the error rate was 2-5% and peaked at 5% of requests to the service. This was due to an observability misconfiguration. We mitigated the incident by updating the observability configuration and redeploying. We are working to reduce our time to detection and mitigation of issues like this one in the future.
Jul 03, 4:40 PM UTC
minor
Mitigation measures have been rolled out and we're seeing errors disappear in our telemetry. We'll continue to monitor our services closely to ensure the docs site is fully healthy.
Jul 03, 4:37 PM UTC
minor
We have identified a likely cause of the errors with GitHub Docs and are working on a mitigation.
Jul 03, 3:59 PM UTC
minor
We are currently investigating this issue.
Jul 03, 3:24 PM UTC
good
This incident has been resolved.
Jul 02, 7:24 PM UTC
minor
API Requests is operating normally.
Jul 02, 7:24 PM UTC
minor
The fix has been rolled out and our telemetry indicates that the errors with code search have resolved.
Jul 02, 7:22 PM UTC
minor
An issue with faulty data in an in-memory storage causes around one third of code search requests to fail. The team has identified the issue and is working on rolling out a fix.
Jul 02, 7:18 PM UTC
minor
API Requests is experiencing degraded performance. We are continuing to investigate.
Jul 02, 6:47 PM UTC
minor
We are currently investigating this issue.
Jul 02, 6:45 PM UTC
good
At approximately 19:20 UTC on July 1st, 2024, one of GitHub’s peering links to a public cloud provider began experiencing 5 - 20% packet loss. This resulted in intermittent network timeouts running Git operations for customers who run their own environments with that specific provider.

Investigation pointed to an issue with the physical link. At 01:14 UTC we rerouted traffic away from the problematic link to other connections to resolve the incident.

Jul 02, 1:14 AM UTC
minor
As a result of rerouting traffic, we have seen overall network link health return to normal.
Jul 02, 1:14 AM UTC
minor
We are investigating connection issues with one of our network links. We are working to reroute traffic.
Jul 02, 12:52 AM UTC
minor
We are investigating intermittent network connection issues. These issues appear to be limited to customers hosted on AWS that are connecting to GitHub's network.
Jul 02, 12:15 AM UTC
minor
We're investigating reports of intermittent timeouts and connection errors for git clone operations.
Jul 01, 11:32 PM UTC
minor
We are investigating reports of degraded performance for Git Operations
Jul 01, 10:59 PM UTC
good
On June 28th, 2024, at 16:06 UTC, a backend update by GitHub triggered a significant number of long-running Organization membership update jobs in our job processing system. The job queue depth rose as these update jobs consumed most of our job worker capacity. This resulted in delays for other jobs across services such as Pull Requests and PR-related Actions workflows. We mitigated the impact to Pull Requests and Actions at 19:32 UTC by pausing all Organization membership update jobs. We deployed a code change at 22:30 UTC to skip over the jobs queued by the backend change and re-enabled Organization membership update jobs. We restored the Organization membership update functionality at 22:52 UTC, including all membership changes queued during the incident.

During the incident, about 15% of Action workflow runs experienced a delay of more than five minutes. In addition, Pull Requests had delays in determining merge eligibility and starting associated Action workflows for the duration of the incident. Organization membership updates saw delays for upwards of five hours.

To prevent a similar event in the future from impacting our users, we are working to: improve our job management system to better manage our job worker capacity; add more precise monitoring for job delays; and strengthen our testing practices to prevent future recurrences.
Jun 28, 10:51 PM UTC
minor
We are continuing to work to mitigate delays in organization membership changes.
Jun 28, 10:18 PM UTC
minor
We are still actively working to mitigate delays in organization membership changes.
Jun 28, 9:45 PM UTC
minor
We are actively working to mitigate delays in organization membership changes. Actions and Pull Requests are both functioning normally now.
Jun 28, 8:46 PM UTC
minor
Actions is operating normally.
Jun 28, 8:00 PM UTC
minor
Pull Requests is operating normally.
Jun 28, 7:59 PM UTC
minor
We are continuing to apply mitigations and are seeing improvement in creating pull request merge commits and Actions runs for pull request events. Applying changes to organization members remains delayed.
Jun 28, 7:51 PM UTC
minor
We are continuing to work on mitigating delays creating pull request merge commits, Actions runs for pull request events, and changes to organization members.
Jun 28, 7:03 PM UTC
minor
Actions runs triggered by pull requests are experiencing start delays. We have engaged the appropriate teams and are investigating the issue.
Jun 28, 5:59 PM UTC
minor
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Jun 28, 5:58 PM UTC
minor
We are investigating reports of degraded performance for Actions
Jun 28, 5:34 PM UTC
good
This incident has been resolved.
Jun 27, 11:44 PM UTC
minor
We have identified a root cause for Codespaces issue in the West US region and are rolling out a fix.
Jun 27, 11:43 PM UTC
minor
A subset of customers are currently experiencing issues creating and resuming Codespaces in the West US region.
Jun 27, 11:34 PM UTC
minor
We are investigating reports of degraded performance for Codespaces
Jun 27, 11:34 PM UTC
good
Between June 27th, 2024 at 20:39 UTC and 21:37 UTC the Migrations service was unable to process migrations. This was due to a invalid infrastructure credential.

We mitigated the issue by updating the credential and deploying the service.

Mechanisms and automation will be implemented to detect and prevent this issue again in the future.
Jun 27, 9:42 PM UTC
major
Some GitHub Enterprise Importer migrations are failing. We have identified a root cause and are rolling out a fix.
Jun 27, 9:20 PM UTC
major
We are currently investigating this issue.
Jun 27, 9:16 PM UTC
good
Between June 18th, 2024 at 09:34 PM UTC and June 19th, 2024 at 12:53 PM the Copilot Pull Request Summaries Service was unavailable. This was due to an internal change in access approach from the Copilot Pull Request service to the Copilot API.

We mitigated the incident by reverting the change in access which immediately resolved the errors.

We are working to improve our monitoring in this area and reduce our time to detection to more quickly address issues like this one in the future.

Jun 19, 12:53 PM UTC
minor
We are deploying a fix now and expect recovery within the hour.
Jun 19, 12:31 PM UTC
minor
We’ve identified an issue with Copilot pull request summaries that has caused errors when attempting to generate summaries since yesterday (June 18, 2024) at around 21:00 UTC.

We have identified a fix, and we expect the issue to be resolved within two hours.
Jun 19, 11:59 AM UTC
minor
We are investigating reports of degraded performance for Copilot
Jun 19, 11:58 AM UTC
good
Starting on June 18th from 4:59pm UTC to 6:06pm UTC, customer migrations were unavailable and failing. This impacted all in-progress migration during that time. This issue was due to an incorrect configuration on our Database cluster. We mitigated the issue by remediating the database configuration and are working with stakeholders to ensure safeguards are in place to prevent the issue going forward.
Jun 18, 6:09 PM UTC
minor
We have applied a configuration change to our migration service as a mitigation and are beginning to see recovery and in increase in successful migration runs. We are continuing to monitor.
Jun 18, 6:04 PM UTC
minor
We have identified what we believe to be the source of the migration errors and are applying a mitigation, which we expect will begin improving migration success rate.
Jun 18, 5:48 PM UTC
minor
We are investigating degraded performance for GitHub Enterprise Importer migrations. Some customers may see an increase in failed migrations. Investigation is ongoing.
Jun 18, 5:15 PM UTC
minor
We are currently investigating this issue.
Jun 18, 5:14 PM UTC
good
On June 11th, 2024 between 20:13 UTC and 21:39 UTC, the GitHub Actions service was degraded. A security-related change applied by one of our third-party providers prevented new customers from onboarding to GitHub Actions and caused an average 28% of Actions jobs to fail.

We mitigated the incident by working with the third-party provider to revert the change and are working with their engineering team to fully understand the root cause. Additionally, we are improving communication between GitHub and our service providers to reduce the time needed to resolve similar issues in the future.
Jun 11, 9:39 PM UTC
minor
We've applied a mitigation to unblock running Actions and are seeing an improvement in our service availability.
Jun 11, 9:35 PM UTC
minor
Customers may see issues running Actions, we are in the process of applying a mitigation to restore our service.
Jun 11, 9:16 PM UTC
minor
Customers may see issues running Actions
Jun 11, 8:34 PM UTC
minor
We are investigating reports of degraded performance for Actions and API Requests
Jun 11, 8:33 PM UTC
good
On June 6, 2024 between 03:29 and 04:19 UTC, the service responsible for the Maven package registry was degraded. This affected GitHub customers who were trying to upload packages to the Maven package registry.

We observed increased database pressure due to bulk operations in progress, and at 04:19 UTC, the Maven upload issues resolved when those bulk operations finished. We're continuing to assess any additional compounding factors.

We are working on improving our thresholds for existing alerts to reduce our time to detection and mitigation of issues like this one in the future.

Jun 06, 4:43 AM UTC
minor
We were alerted to problems in maven uploads. These have now improved, and we're continuing to monitor and investigate.
Jun 06, 4:38 AM UTC
minor
We are investigating reports of issues with Packages. We will continue to keep users updated on progress towards mitigation.
Jun 06, 4:21 AM UTC
minor
We are investigating reports of degraded performance for Packages
Jun 06, 4:21 AM UTC
good
On June 5, 2024, between 17:05 UTC and 19:27 UTC, the GitHub Issues service was degraded. During that time, no events related to projects were displayed on issue timelines. These events indicate when an issue was added to or removed from a project and when their status changed within a project. The data couldn’t be loaded due to a misconfiguration of the service backing these events. This happened after a scheduled secret rotation when the wrongly configured service continued using the old secrets which had expired.

We mitigated the incident by remediating the service configuration and have started simplifying the configuration to avoid similar misconfigurations in the future.
Jun 05, 7:27 PM UTC
major
Issues is operating normally.
Jun 05, 7:27 PM UTC
major
We continue to troubleshoot the problem with issues timeline.
Jun 05, 7:19 PM UTC
major
We continue to troubleshoot the problem with issues timeline.
Jun 05, 6:47 PM UTC
major
We're continuing to investigate the problem.
Jun 05, 6:01 PM UTC
major
We're seeing issues related to the issues timeline service, we're investigating and we will continue to keep users updated on progress towards mitigation.
Jun 05, 5:26 PM UTC
major
We are investigating reports of degraded availability for Issues
Jun 05, 5:22 PM UTC
good
On May 30th, 2024, between 03:37 PM UTC and 05:14 PM UTC Copilot chat conversations on github.com saw degraded availability, where chat requests referencing files from a repository failed. This was due to an expired security certificate, which required communication to an internal service. Overall, the error rate was 40% on average. Other Copilot chat experiences were unaffected during this time.

The incident was mitigated by rotating the certificate in question.

To prevent future incidents, we are working to reduce our time to detect and have removed certificate-based dependencies between these internal systems in the process.

May 30, 5:22 PM UTC
minor
Copilot is operating normally.
May 30, 5:22 PM UTC
minor
We have rolled out mitigation and fixes appear to be stable. This incident has been resolved.
May 30, 5:22 PM UTC
minor
Copilot is experiencing degraded performance. We are continuing to investigate.
May 30, 5:14 PM UTC
minor
Our CoPilot API is currently experiencing back-end connectivity issues and we are actively engaged in mitigation steps.
May 30, 5:14 PM UTC
minor
We are currently investigating this issue.
May 30, 5:14 PM UTC
minor
Codespaces is operating normally.
May 28, 9:24 PM UTC
good
This incident has been resolved.
May 28, 9:24 PM UTC
minor
A fix has been applied and we are seeing some recovery. We will continue to monitor for a bit before marking this issue resolved.
May 28, 9:19 PM UTC
minor
We are still investigating root cause and remediation options. In the meantime, here is a workaround to be able to pull images from DockerHub:

1. Make a free DockerHub account at https://hub.docker.com (or use an existing account if you have one).
2. Create a DockerHub secret/PAT from https://hub.docker.com/settings/security (Read permission should be sufficient).
3. Go to https://github.com/settings/codespaces

Add three Codespace secrets:

- DOCKERHUB_CONTAINER_REGISTRY_PASSWORD (equal to the DockerHub PAT you created)
- DOCKERHUB_CONTAINER_REGISTRY_SERVER (equal to https://index.docker.io/v1/)
- DOCKERHUB_CONTAINER_REGISTRY_USER (equal to your DockerHub username)

4. Make sure these secrets are set as visible to the target repo.
5. Create/rebuild your Codespace

Steps above are distilled from the official docs: https://docs.github.com/en/codespaces/reference/allowing-your-codespace-to-access-a-private-registry#example-secrets
May 28, 8:53 PM UTC
minor
Duplicate update, same as above
May 28, 8:53 PM UTC
minor
Some Codespaces are currently failing to be properly created for images hosted by DockerHub. Other registries should be unaffected. We are investigating root cause and will report back shortly.
May 28, 8:23 PM UTC
minor
We are investigating reports of degraded performance for Codespaces
May 28, 8:17 PM UTC
good
On May 23, 2024 between 15:31 and 16:02 the Codespaces service reported a degraded experience in codespaces across all regions. Upon further investigation this was found to be an error reporting issue and did not have user facing impact. The new error reporting that was implemented began raising on existing non-user facing errors that are handled further in the flow, at the controller level, which do not cause user impact. We are working to improve our reporting roll out process to reduce issues like this in the future which includes updating monitors and dashboards to exclude this class of error. We are also reclassifying and correcting internal API responses to better represent when errors are user facing for more accurate reporting.
May 23, 4:02 PM UTC
minor
We are investigating increased error rates for customers attempting to start Codespaces across all regions, around 15% of attempts are affected. Any affected customers may attempt to retry starting their Codespace. We are continuing to investigate.
May 23, 3:41 PM UTC
minor
We are investigating reports of degraded performance for Codespaces
May 23, 3:31 PM UTC
good
On May 21, 2024, between 11:40 UTC and 19:06 UTC various services experienced elevated latency due to a configuration change in an upstream cloud provider.

GitHub Copilot Chat experienced P50 latency of up to 2.5s and P95 latency of up to 6s. GitHub Actions was degraded with 20 - 60 minute delays for workflow run updates. GitHub Enterprise Importer customers experienced longer migration run times due to GitHub Actions delays. Additionally, billing related metrics for budget notifications and UI reporting were delayed leading to outdated billing details. No data was lost and systems caught up after the incident.

At 12:31 UTC, we detected increased latency to cloud hosts. At 14:09 UTC, non-critical traffic was paused, which did not result in restoration of service. At 14:27 UTC, we identified high CPU load within a network gateway cluster caused by a scheduled operating system upgrade that resulted in unintended, uneven distribution of traffic within the cluster. We initiated deployment of additional hosts at 16:35 UTC. Rebalancing completed by 17:58 UTC with system recovery observed at 18:03 UTC and completion at 19:06 UTC.

We have identified gaps in our monitoring and alerting for load thresholds. We have prioritized these fixes to improve time to detection and mitigation of this class of issues.
May 21, 7:06 PM UTC
minor
Actions is operating normally.
May 21, 6:14 PM UTC
minor
We are beginning to see recovery for any delays to Actions Workflow Runs, Workflow Job Runs, and Check Steps. Customers who are still experiencing jobs which appear to be stuck may re-run the workflow in order to see a completed state. We are also seeing recovery for GitHub Enterprise Importer migrations. We are continuing to monitor recovery.
May 21, 6:03 PM UTC
minor
We are continuing to investigate delays to status updates to Actions Workflow Runs, Workflow Job Runs, and Check Steps. This is impacting 100% of customers using these features, with an average delay of 20 minutes and P99 delay of 1 hour. Customers may see that their Actions workflows may have completed, but the run may appear to be hung waiting for its status to update. This is also impacting GitHub Enterprise Importer migrations. Migrations may take longer to complete. We are are working with our provider to address the issue and will continue to provide updates as we learn more.
May 21, 5:41 PM UTC
minor
We are continuing to investigate delays to status updates to Actions Workflow Runs, Workflow Job Runs, and Check Steps. Customers may see that their Actions workflows may have completed, but the run may appear to be hung waiting for its status to update. This is also impacting GitHub Enterprise Importer migrations. Migrations may take longer to complete. We are are working with our provider to address the issue and will continue to provide updates as we learn more.
May 21, 5:14 PM UTC
minor
We are continuing to investigate delays to Actions Workflow Runs, Workflow Job Runs, and Check Steps and will provide further updates as we learn more.
May 21, 4:02 PM UTC
minor
We have identified a change in a third party network configuration and are working with the provider to address the issue. We will continue to provide updates as we learn more.
May 21, 3:00 PM UTC
minor
We have identified network connectivity issues causing delays in Actions Workflow Runs, Workflow Job Runs, and Check Steps. We are continuing to investigate.
May 21, 2:34 PM UTC
minor
We are investigating delayed updates to Actions job statuses.
May 21, 1:58 PM UTC
minor
We are investigating reports of degraded performance for Actions
May 21, 12:45 PM UTC
good
Between May 19th 3:40AM UTC and May 20th 5:40PM UTC the service responsible for rendering Jupyter notebooks was degraded. During this time customers were unable to render Jupyter Notebooks.

This occurred due to an issue with a Redis dependency which was mitigated by restarting. An issue with our monitoring led to a delay in our response. We are working to improve the quality and accuracy of our monitors to reduce the time to detection.
May 20, 5:05 PM UTC
minor
We are beginning to see recovery rendering Jupyter notebooks and are continuing to monitor.
May 20, 5:01 PM UTC
minor
Customers may experience errors viewing rendered Jupyter notebooks from PR diff pages or the files tab
May 20, 4:50 PM UTC
minor
We are currently investigating this issue.
May 20, 4:47 PM UTC