In Acquia Node.js Hosting, memory limit of 1.8GB is assigned to each Node.js process. If a process tries to claim more than the assigned memory, the platform restarts the Node.js process forcefully. It usually happens because of memory leak in the application.
This document provides a series of necessary steps to identify, troubleshoot, and resolve memory leaks in Node.js applications hosted on Cloud Platform. While it does not focus on troubleshooting performance issues in JavaScript applications, the methods presented may also assist in addressing these concerns.
Identifying memory leaks
The following are the methods to identify memory leaks in Node.js applications hosted on Cloud Platform:
Locate the Out of memory errors section and review the instances where OOM errors have caused application restarts.
Data points that exceed 0 indicate the time when the Node.js application restarted because of OOM errors.
Locate the Memory usage section and track the application memory usage.
Monitor for a persistent rise in the memory usage that remains high even when the application is idle.
For example, the following image shows that the memory usage gradually increases to nearly 70% before being reset or terminated by the Linux system an because of an OOM event. The process then restarts and is terminated once more upon reaching the memory constraints.
The memory usage shown corresponds to the server's capacity. Therefore, it does not reach 100% because memory is still available in the system.
Application Performance Management (APM) tools such as New Relic provide significant assistance in diagnosing memory leaks. Node.js customers receive complimentary access to a New Relic instance for performance monitoring.
Log in to New Relic and click APM & Services > Node VMs.
Select the VM instance of your application.
For example, select container_rootfs in the preceding image.
Examine metrics for the VM instance:
GC pause time
During the Garbage Collection (GC) phase, the application halts and does not process any requests. If the pause time is longer, it indicates that the application is dedicating more time to cleanup tasks. With a memory leak, the garbage collector does not reclaim memory effectively even after substantial GC activity. Significant and recurring spikes in GC pause time suggest a memory leak, which negatively affects the application's performance.
Note
GC pause time refers to the period during which the runtime system is engaged in collecting garbage.
GC pause frequency
An increased frequency of GC pauses suggests more frequent garbage collections, which may point to a memory leak.
Note
GC pause frequency refers to the occurrence count of garbage collector invocations within a given period.
Memory usage (MB)
In a well-functioning application, the garbage collector releases memory each time it activates. If the memory is not released at consistent intervals, it suggests a memory leak. In the following chart, the freed-up memory is gradually decreasing after every GC event.
Preventing memory leaks through logging
Logging can be employed to monitor and prevent memory leaks. For example:
Utilize the inherent process.memoryUsage() function of Node.js to systematically record memory utilization statistics.
Document metrics such as heapTotal, heapUsed, and rss.
When substantial proofs of memory leaks exist, follow these instructions to address them.
Investigating and resolving Out of memory errors
If you experience Out of memory errors, you can take several proactive steps to investigate and resolve these errors.
Follow these suggestions to conserve memory resources and prevent future issues:
Investigate memory consumption: Use relevant documentation and tools to identify major memory-consuming processes and pinpoint potential causes of memory exhaustion errors. Understanding of these causes helps you to apply targeted solutions.
Reduce the Cron job frequency: Cron jobs significantly impact memory usage, especially if they are scheduled too frequently. Therefore, minimize the number of cron jobs and adjust their frequency to the absolute minimum setting that still supports operations of your application. Check your system's instructions for disabling or modifying cron jobs as needed.
Adjust the PHP memory limit: The PHP memory limit setting significantly influences application performance. Therefore, configure an adequate PHP memory limit for each environment so that the application loads successfully. Check your platform's guidelines to adjust the PHP memory limit accordingly.
Avoid unconditional memory limit overrides: While increasing memory limits might seem like a solution, unconditional overrides can negatively impact your infrastructure by overriding system-wide constraints. Therefore, ensure that all memory limit changes are purposeful and conditional based on specific requirements rather than blanket increases.
Additional resources and assistance: For further guidance or questions, consult the additional documentation provided or contact your Support team for assistance in implementing these actions.
These suggestions are designed to provide an outline to manage and resolve memory-related issues effectively and sustainably.
Employ Chrome DevTools or the Node.js --inspect flag to capture heap snapshots.
Examine snapshots from different intervals to identify the objects that are not subject to garbage collection.
Analyze Heap snapshots:
Check the objects that are increasing in size over time.
Be vigilant of detached DOM nodes, untracked listeners, or escalating data structures.
Utilize profiling tools:
Apply tools such as clinic.js (formerly node-clinic) or 0x to profile your application and pinpoint memory leaks.
Employ v8-profiler-next or node-inspect for a more in-depth analysis.
Identify common sources, such as:
Global variables or closures retaining references.
Persistent event listeners.
Indefinitely expanding caches.
Database connections or file handles that remain open.
Revise the code:
After pinpointing the leak's origin, alter your code to facilitate memory release.
Confirm the removal of unnecessary event listeners.
Incorporate weak references (WeakMap, WeakSet) for caches when fitting.
Enhance data structures:
Adopt more efficient data structures to avoid unnecessary retention of large objects.
Consider streaming data processing for extensive data sets.
Reorganize for improved memory management:
Divide extensive functions or modules into smaller, more digestible segments.
Incorporate design patterns conducive to adept memory utilization, such as the Flyweight pattern.
Conduct rigorous testing:
Verify the resolution of the memory leak by testing your application under simulated operating conditions after modifications.
Implement automated tests to mimic actual usage scenarios.
Mitigate future memory leaks:
Code reviews:
Implement a code review process to catch potential memory leaks before they reach production.
Continuous monitoring:
Set up continuous monitoring to detect memory issues early.
Use alerts to notify you of unusual memory usage patterns.
Educate the team:
Train your team on best practices for memory management in Node.js applications.
Share knowledge about common pitfalls and how to avoid them.
With these instructions, you can systematically detect, diagnose, and rectify memory leaks within your Node.js application, and obtain enhanced stability and efficiency.
Memory leaks
In Acquia Node.js Hosting, memory limit of 1.8GB is assigned to each Node.js process. If a process tries to claim more than the assigned memory, the platform restarts the Node.js process forcefully. It usually happens because of memory leak in the application.
This document provides a series of necessary steps to identify, troubleshoot, and resolve memory leaks in Node.js applications hosted on Cloud Platform. While it does not focus on troubleshooting performance issues in JavaScript applications, the methods presented may also assist in addressing these concerns.
Identifying memory leaks
The following are the methods to identify memory leaks in Node.js applications hosted on Cloud Platform:
Locate the Out of memory errors section and review the instances where OOM errors have caused application restarts.
Data points that exceed 0 indicate the time when the Node.js application restarted because of OOM errors.
Locate the Memory usage section and track the application memory usage.
Monitor for a persistent rise in the memory usage that remains high even when the application is idle.
For example, the following image shows that the memory usage gradually increases to nearly 70% before being reset or terminated by the Linux system an because of an OOM event. The process then restarts and is terminated once more upon reaching the memory constraints.
The memory usage shown corresponds to the server's capacity. Therefore, it does not reach 100% because memory is still available in the system.
Application Performance Management (APM) tools such as New Relic provide significant assistance in diagnosing memory leaks. Node.js customers receive complimentary access to a New Relic instance for performance monitoring.
Log in to New Relic and click APM & Services > Node VMs.
Select the VM instance of your application.
For example, select container_rootfs in the preceding image.
Examine metrics for the VM instance:
GC pause time
During the Garbage Collection (GC) phase, the application halts and does not process any requests. If the pause time is longer, it indicates that the application is dedicating more time to cleanup tasks. With a memory leak, the garbage collector does not reclaim memory effectively even after substantial GC activity. Significant and recurring spikes in GC pause time suggest a memory leak, which negatively affects the application's performance.
Note
GC pause time refers to the period during which the runtime system is engaged in collecting garbage.
GC pause frequency
An increased frequency of GC pauses suggests more frequent garbage collections, which may point to a memory leak.
Note
GC pause frequency refers to the occurrence count of garbage collector invocations within a given period.
Memory usage (MB)
In a well-functioning application, the garbage collector releases memory each time it activates. If the memory is not released at consistent intervals, it suggests a memory leak. In the following chart, the freed-up memory is gradually decreasing after every GC event.
Preventing memory leaks through logging
Logging can be employed to monitor and prevent memory leaks. For example:
Utilize the inherent process.memoryUsage() function of Node.js to systematically record memory utilization statistics.
Document metrics such as heapTotal, heapUsed, and rss.
When substantial proofs of memory leaks exist, follow these instructions to address them.
Investigating and resolving Out of memory errors
If you experience Out of memory errors, you can take several proactive steps to investigate and resolve these errors.
Follow these suggestions to conserve memory resources and prevent future issues:
Investigate memory consumption: Use relevant documentation and tools to identify major memory-consuming processes and pinpoint potential causes of memory exhaustion errors. Understanding of these causes helps you to apply targeted solutions.
Reduce the Cron job frequency: Cron jobs significantly impact memory usage, especially if they are scheduled too frequently. Therefore, minimize the number of cron jobs and adjust their frequency to the absolute minimum setting that still supports operations of your application. Check your system's instructions for disabling or modifying cron jobs as needed.
Adjust the PHP memory limit: The PHP memory limit setting significantly influences application performance. Therefore, configure an adequate PHP memory limit for each environment so that the application loads successfully. Check your platform's guidelines to adjust the PHP memory limit accordingly.
Avoid unconditional memory limit overrides: While increasing memory limits might seem like a solution, unconditional overrides can negatively impact your infrastructure by overriding system-wide constraints. Therefore, ensure that all memory limit changes are purposeful and conditional based on specific requirements rather than blanket increases.
Additional resources and assistance: For further guidance or questions, consult the additional documentation provided or contact your Support team for assistance in implementing these actions.
These suggestions are designed to provide an outline to manage and resolve memory-related issues effectively and sustainably.
Employ Chrome DevTools or the Node.js --inspect flag to capture heap snapshots.
Examine snapshots from different intervals to identify the objects that are not subject to garbage collection.
Analyze Heap snapshots:
Check the objects that are increasing in size over time.
Be vigilant of detached DOM nodes, untracked listeners, or escalating data structures.
Utilize profiling tools:
Apply tools such as clinic.js (formerly node-clinic) or 0x to profile your application and pinpoint memory leaks.
Employ v8-profiler-next or node-inspect for a more in-depth analysis.
Identify common sources, such as:
Global variables or closures retaining references.
Persistent event listeners.
Indefinitely expanding caches.
Database connections or file handles that remain open.
Revise the code:
After pinpointing the leak's origin, alter your code to facilitate memory release.
Confirm the removal of unnecessary event listeners.
Incorporate weak references (WeakMap, WeakSet) for caches when fitting.
Enhance data structures:
Adopt more efficient data structures to avoid unnecessary retention of large objects.
Consider streaming data processing for extensive data sets.
Reorganize for improved memory management:
Divide extensive functions or modules into smaller, more digestible segments.
Incorporate design patterns conducive to adept memory utilization, such as the Flyweight pattern.
Conduct rigorous testing:
Verify the resolution of the memory leak by testing your application under simulated operating conditions after modifications.
Implement automated tests to mimic actual usage scenarios.
Mitigate future memory leaks:
Code reviews:
Implement a code review process to catch potential memory leaks before they reach production.
Continuous monitoring:
Set up continuous monitoring to detect memory issues early.
Use alerts to notify you of unusual memory usage patterns.
Educate the team:
Train your team on best practices for memory management in Node.js applications.
Share knowledge about common pitfalls and how to avoid them.
With these instructions, you can systematically detect, diagnose, and rectify memory leaks within your Node.js application, and obtain enhanced stability and efficiency.
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Node JS