This incident has been resolved.
We found some additional webhooks that were not a part of our re-send on 5/30 ~4:30pm ET that were subsequently sent on 5/31 at ~11:30pm ET because they were deemed safe to resend. These webhooks were for subscriptions that had not generated additional webhooks after 5/30, so they contain the most up-to-date information.
We found a much smaller group of webhooks that were not re-sent that were deemed not safe to resend. A customer support representative will be reaching out to the affected merchants.
May 30, 15:35 CDT
All webhooks have been successfully resent and this incident has been resolved. Please contact support if you have any questions or issues.
May 30, 15:34 CDT
There have been some merchant-defined webhooks that have failed due to the expired certificate being returned in the CA bundle from those merchant-controlled servers. We have updated our client code to ignore that expired certificate and will be re-sending the webhooks.
There have also been some internal webhooks related to our Salesforce v1, QBO, and Xero integrations that have failed due to the certificate issues. We will be re-sending those as well, so any delays you currently see in your sync to those third party apps will be resolved shortly.
May 30, 14:43 CDT
A fix has been implemented to the CA bundle on the server-side, and we are monitoring the results.
May 30, 13:34 CDT
You may experience problems accessing the Chargify API via a client that does not have updated trusted root certificates.
Chargify is in the process of attempting to replace an intermediate certificate which may, for some clients, force them to use a more recent (and unexpired) root that they already trust.
In the meantime, Chargify merchants can fix this for themselves by adding more recent root certificates to their trust stores. Add either one of these to the certificate store used by your client:https://crt.sh/?id=1199354https://crt.sh/?id=1720081
Note the "Download Certificate: PEM" link on the above webpages to obtain the certificate.
For example, if you are using curl as your client, you can specify a certificate authority file to use:
curl --cacert /etc/ssl/1199354.crt -u API_KEY:X https://subdomain.chargify.com/subscriptions.json
The above is an example. How to install or use the updated certificate is dependent on your client.
You can learn more here: https://support.sectigo.com/Com_KnowledgeDetailPage?Id=kA03l00000117LT
May 30, 11:02 CDT
An upstream SSL Signing Certificate has expired and we are actively working to redeploy our SSL Certificates to exclude this signing certificate from our bundle. The issue shouldn't affect browsers but will affect curl and openssl clients. Merchants can update their clients to resolve the issue while we work to redeploy our certificates.
May 30, 10:18 CDT
We are currently investigating reports of a potential service interruption with the Chargify API. We apologize for any inconvenience and will post another update as soon as we learn more.
May 30, 09:57 CDT