Performance troubleshooting is ultimately a process of elimination. It is important to try to eliminate the biggest factors first (e.g. environment) while getting more granular as you proceed (e.g. code execution). Understanding recent changes to an environment is critical to identify where to look first. For example, a brand new environment poses more possible sources of performance problems than an environment where you've deployed multiple times and only the most recent deploy has coincided with the performance issue.
Recently we had to triage a performance issue in a Sitecore build. After going through the exercise, I thought I would share some of our troubleshooting steps and thoughts. These diagnostic checks should help to eliminate some common areas where performance issues might be found.
Rule out the specific page or template
Test multiple pages as well as pages based on different templates to confirm you are seeing the same behaviour across different page types.
Test different instances/environments to see if your issue is present
In a scaled environment, confirm if you see the issue on both the content author and content delivery boxes. Additionally, see if you can reproduce the issue in a non-production environment such as your pre-production and/or development environment
Add a test page to the build that has no calls to Sitecore
If you reproduce the issue in this scenario, it is indicative that the issue is not code related.
Load the Sitecore heart beat page
Since it does call the Sitecore API and subsequently hits the database, it helps to identify if there is a database issue and it can help identify your code as an issue if no performance issue is found.
Eliminate a specific component as the cause
Create a test page by duplicating an existing page and start removing components
Granted, this can be tricky in a production environment, but the test attempts to identify a specific component as the culprit. If your page marginally increase performance as you remove components, then it is likely not an issue with a specific component
Use Sitecore's debug mode
Look for a component that takes a long time to run or has excessive item counts
Use Google developer tools
The Network tab on the Google developer tools can tell you how long it takes the server to respond to your request. For more information: https://developers.google.com/web/tools/chrome-devtools/profile/network-performance/resource-loading?hl=en
Disable analytics/xDB temporarily
Sitecore collects a lot of information when a page loads. Disabling that collection may identify data collection as a possible bottleneck potentially as the result of wrong configuration or a slow communication channel.
Review the Sitecore pipeline profiler
It is common to inject custom pipelines in Sitecore, so the ability to monitor the performance of custom and out-of-the-box pipelines is crucial. You can do that as a Sitecore administrator at /sitecore/admin/pipelines.aspx. You can find more information here:
Revert to the stock Sitecore Web.config and App_Config settings
Obviously, you wouldn't want to do this in a production environment without ensuring you are not impacting your site traffic, but removing all the custom configurations and settings can help identify if your problem is configuration related. If you issue resolves itself at this step, reintroduce your configurations in small steps to help identify the specific configuration patch.
Do a file comparison
It is possible that a deployment did not work as expected and old or missing file(s) can be the root cause of your issue