-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expunge stale loggers in InternalLoggerRegistry on method invocation #3474
base: 2.x
Are you sure you want to change the base?
Conversation
…so added InternalLoggerRegistryGCTest
@vy I have always had a hard time with test cases, I have opened a pr regardless please take look and provide further insights on how i can improve it. |
...j-core/src/main/java/org/apache/logging/log4j/core/util/internal/InternalLoggerRegistry.java
Outdated
Show resolved
Hide resolved
...j-core/src/main/java/org/apache/logging/log4j/core/util/internal/InternalLoggerRegistry.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please put back all the removed comment blocks.
Ok sorry about that, I have put back all the comment blocks as they were. |
...j-core/src/main/java/org/apache/logging/log4j/core/util/internal/InternalLoggerRegistry.java
Outdated
Show resolved
Hide resolved
...j-core/src/main/java/org/apache/logging/log4j/core/util/internal/InternalLoggerRegistry.java
Outdated
Show resolved
Hide resolved
Map<String, WeakReference<Logger>> loggerRefByName = loggerRefByNameByMessageFactory.get(messageFactory); | ||
if (loggerRefByName != null) { | ||
loggerRefByName.remove(name); | ||
if (loggerRefByName.isEmpty()) { | ||
loggerRefByNameByMessageFactory.remove(messageFactory); // Cleanup | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assume I move this logic (and only this!) to a method called unsafeRemoveLogger(Logger)
. Note that this method receives a Logger
, not a Reference
.
Reference<? extends Logger> loggerRef; | ||
while ((loggerRef = staleLoggerRefs.poll()) != null) { | ||
removeLogger(loggerRef); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind changing this loop as follows, please?
boolean locked = false;
Reference<? extends Logger> loggerRef;
while ((loggerRef = staleLoggerRefs.poll()) != null) {
Logger logger = loggerRef.get();
if (logger != null) {
if (!locked) {
writeLock.lock();
}
unsafeRemoveLogger(logger);
}
}
if (locked) {
writeLock.unlock();
}
See my other comment regarding the unsafeRemoveLogger(Logger)
method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that if loggerRef.get()
returns null
, we still need to remove the entry that holds the WeakReference
.
What do you think about such an approach:
- We
poll()
the queue until its empty. - If there was any reference in the queue we iterate once over all the entries of the map and we remove all the invalidated
WeakReference
s, not just those that were in the queue.
As of my previous comment, this seems a little bit inefficient if loggerRef.get()
is not null, but is more efficient than iterating over all entries once-per-reference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ppkarwasz, you're right. @Suvrat1629, please proceed with the suggestion of @ppkarwasz. Though I suggest having double-checking:
if (!refQueue.isEmpty) {
lock();
try {
if (!refQueue.isEmpty) {
// Clear `refQueue`
// Clean up `loggerRefByNameByMessageFactory`
}
} finally {
unlock();
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you replace LoggerContext.getContext()
calls as follows, please?
@Test
void someTest(TestInfo testInfo) {
try (final LoggerContext loggerContext = new LoggerContext(testInfo.getDisplayName)) {
// Use `loggerContext`
}
}
That is, don't disrupt the global one, use ephemeral LC
s instead.
Logger logger = | ||
registry.computeIfAbsent("testLogger", messageFactory, (name, factory) -> LoggerContext.getContext() | ||
.getLogger(name, factory)); | ||
|
||
WeakReference<Logger> weakRef = new WeakReference<>(logger); | ||
logger = null; // Dereference to allow GC | ||
|
||
// Retry loop to give GC time to collect | ||
for (int i = 0; i < 10; i++) { | ||
System.gc(); | ||
Thread.sleep(100); | ||
if (weakRef.get() == null) { | ||
break; | ||
} | ||
} | ||
|
||
// Access the registry to potentially trigger cleanup | ||
registry.computeIfAbsent("tempLogger", messageFactory, (name, factory) -> LoggerContext.getContext() | ||
.getLogger(name, factory)); | ||
|
||
assertNull(weakRef.get(), "Logger should have been garbage collected"); | ||
assertNull( | ||
registry.getLogger("testLogger", messageFactory), "Stale logger should be removed from the registry"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we instead replace this as follows, please?
- Use an ephemeral
LoggerContext
(as I described in another comment) - Create 1000
Logger
s (should be many enough to create GC pressure) - Log using all
Logger
s (so we imitate a real-world setup whereLogger
s are used at some point) - Use
Awaitility.waitAtMost(...).until(...)
to verify that after aSystem.gc()
,InternalLoggerRegistry::loggerRefByNameByMessageFactory
is emptied
You can access to InternalLoggerRegistry::loggerRefByNameByMessageFactory
using reflection. That is, first extract the LoggerContext::loggerRegistry
field, and then the InternalLoggerRegistry::loggerRefByNameByMessageFactory
field.
This PR updates InternalLoggerRegistry to automatically remove stale loggers when its methods are invoked.
Changes Introduced:
Introduced a ReferenceQueue to track loggers that have been reclaimed by the garbage collector.
Implemented expungeStaleEntries(), which removes stale loggers before executing registry operations.
Updated methods like getLogger() and computeIfAbsent() to invoke expungeStaleEntries() before proceeding.
Added tests to verify stale loggers are expunged without relying on private methods.
Fixes #3430