AI Search
Classic Search
 Search Phrase:
 Search Type:
Advanced search options
 Search in Forums:
 Search in date period:

 Sort Search Results by:

AI Assistant
Notifications
Clear all

Hardcoded 1000-file wipe in Cache::check() — please add a filter

4 Posts
2 Users
0 Reactions
147 Views
sc89me
Posts: 31
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
(@sc89me)
Trusted Member
Joined: 7 years ago
[#73137]

Issue
On a large, busy forum, wpForo’s file cache under {cache_dir}/item/post regularly exceeds 1000 files. Cache::check() then deletes every file in that directory on the same frontend request.

That runs from wp_footer  WPF()->cache->create() when the template is forum (forum index). After the wipe, traffic rebuilds the files, the folder hits 1000 again, and the wipe repeats (for us, every few minutes). PHP CPU spikes and the cache is not useful.

We have thousands of posts. Each cached post is its own file ({postid}_{locale}). A 1000-file cap is far below our working set. There is no setting or filter to change it.

This is still present in 3.1.5. classes/Cache.php check():

if ( $filecount > 1000 ) {
$this->clean_files( $directory );
}

Page cache (WP Rocket etc.) is not in use on the forum; this is only wpForo’s on-disk cache.

Request
Please make the limit filterable, default 1000, pass $directory, and treat 0 as never wipe:

$max = (int) apply_filters( 'wpforo_cache_dir_max_files', 1000, $directory );
if ( $max > 0 && $filecount > $max ) {
$this->clean_files( $directory );
}

That stays backward compatible. Large forums can raise the cap or disable the wipe without forking the plugin.

Optional follow-up
Counting the directory with FilesystemIterator on every forum-index request is expensive once the folder is large, even if you never delete. A throttle (e.g. at most once per N minutes) would help. Full-directory unlink is also harsh vs deleting oldest files; the filter is the minimum we need please.


3 Replies
sc89me
Posts: 31
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
(@sc89me)
Trusted Member
Joined: 7 years ago

We have modified the check function to skip the file count function calls entirely when the max is set to 0 this avoids expensive CPU calls to count large directories which will not be cleared anyway.  

public function check( $directory ) {
		$directory = (string) $directory;
		$directory = wpforo_fix_dir_sep( $directory );
		$filecount = 0;
        $max = (int) apply_filters( 'wpforo_cache_dir_max_files', 1000, $directory );

        if($max === 0) {
            return;
        }

        if( class_exists( 'FilesystemIterator' ) && is_dir( $directory ) ) {
			$fi        = new FilesystemIterator( $directory, FilesystemIterator::SKIP_DOTS );
			$filecount = iterator_count( $fi );
		}
		if( ! $filecount ) {
			$directory_ns = trim( $directory, DIRECTORY_SEPARATOR ) . DIRECTORY_SEPARATOR . '*';
			$directory_ws = DIRECTORY_SEPARATOR . trim( $directory, DIRECTORY_SEPARATOR ) . DIRECTORY_SEPARATOR . '*';
			$files        = glob( $directory_ns );
			if( empty( $files ) ) $files = glob( $directory_ws );
			$filecount = count( $files );
		}

        if ( $filecount > $max ) {
            $this->clean_files( $directory );
        }
}

Filter Used: 

add_filter( 'wpforo_cache_dir_max_files', fn() => 0 );

Reply
Robert
Posts: 10786
Admin
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
(@robert)
Support Team
Joined: 7 months ago

Hi @sc89me,

Thank you for sharing this issue. We'll fix this in the next version release. It'll change the max file number dynamically based on the forum activity and also we'll add filter hook to set a custom value. 


Reply
sc89me
Posts: 31
Topic starter
Translate
English
Spanish
French
German
Italian
Portuguese
Russian
Chinese
Japanese
Korean
Arabic
Hindi
Dutch
Polish
Turkish
Vietnamese
Thai
Swedish
Danish
Finnish
Norwegian
Czech
Hungarian
Romanian
Greek
Hebrew
Indonesian
Malay
Ukrainian
Bulgarian
Croatian
Slovak
Slovenian
Serbian
Lithuanian
Latvian
Estonian
(@sc89me)
Trusted Member
Joined: 7 years ago

Hi Robert, 

Appreciate that thanks, 

Will you please still include a skip (early return) of the function if set to infinite?

When we initially didn't return early and still allowed those file count calls to happen it was heavily loading the CPU because we hit periods of 50 - 100 views per second at peak times which caused those to happen per view. 


Reply
Share: