Throttler usage

[natanfelles]

Hi, I was testing requests by browser to see the throttler work. But do not occurs any 429 HTTP Code response.

Then I created a Command to test this...

<?php namespace App\Commands;

use CodeIgniter\CLI\CLI;
use CodeIgniter\CLI\BaseCommand;

class TestThrottle extends BaseCommand
{
	protected $group       = 'Tests';
	protected $name        = 'test:throttle';
	protected $description = 'Test';

	public function run(array $params)
	{
		CLI::write('Test Throttle', 'light_green');

		$curl = \Config\Services::curlrequest();

		$runtime = 60; // Seconds to continue crawling

		$time    = time();
		$stop_at = $time + $runtime;

		$requests = 1; // The requests counter
		$codes    = []; // HTTP Status Codes and times responded

		while (time() <= $stop_at)
		{
			$response = $curl->get(base_url(), ['http_errors' => false]);

			$code = $response->getStatusCode();

			$codes[$code] = isset($codes[$code]) ? $codes[$code] + 1 : 1;

			switch (substr($code, 0, 1)) {
				case 2:
					$color = 'green';
					break;
				case 3:
					$color = 'blue';
					break;
				case 4:
					$color = 'red';
					break;
				case 5:
					$color = 'yellow';
					break;
				default:
					$color = 'white';
					break;
			}

			$code = CLI::color($code, $color);

			CLI::write('Seconds to end: ' . ($stop_at - time()) . ' - Request ' . $requests++ . ' [' . $code . ']');
			CLI::write('GET: ' . $response->getBody());
			//var_dump($response);exit;
		}

		// Results
		CLI::write('Results', 'light_green');

		CLI::write('Runtime: ' . $runtime . ' seconds');
		CLI::write('Rate of ' . ($requests / $runtime) . ' responses per second');

		foreach ($codes as $code => $times)
		{
			CLI::write("- Code {$code} was responded {$times} times.");
		}
	}

}

To avoid CLI errors after enable the Global Before Filter throttle, I added this in the begging of the before method of the Throttle Filter:

if ($request->isCLI())
{
	return;
}

$throttler = Services::throttler();

And in the Home controller index just an echo date('Y-m-d H:i:s'); to see the body get by the cURL.

Then, testing in the terminal by 60 seconds I had the following result:

Results
Runtime: 60 seconds
Rate of 43.85 responses per second
- Code 200 was responded 2485 times.
- Code 429 was responded 145 times.

By 10 seconds the following:

Runtime: 10 seconds
Rate of 52.8 responses per second
- Code 200 was responded 527 times.

My doubt is: Is "correct" have results like it?

• I used the Files cache driver.

[natanfelles]

The App::baseURL was set to http://localhost:8080/

[lonnieezell]

No, those results don't match what we should expect. Using the default throttler it should allow 1 action per second, after the "bucket" has run out. So it should be doing:

1. Get 60 requests really fast from the same IP. Each request takes one "action" from the bucket, which initially has 60. Since this all happens in less than 1 second, the bucket should now be empty.
2. Attempt another action, but it shouldn't be allowed.
3. At the end of the first second, the bucket should get 1 more "action" added to it, so the next request should proceed.
4. Any other requests should be denied until the next second when the bucket refills with 1 value again.

The tests support this is how it should be working.

I wonder if the IP results you're getting from CLI may be doing strange things since it's testing.

[natanfelles]

First time, now, I thought that could be because the IP get was ::1.

Changing the $request->getIPAddress() call in the Throttle Filter to '127.0.0.1' the result was very similar.

But adding the last param $cost to 60. The results was more faithful:

// entire site.
if ($throttler->check($request->getIPAddress(), 60, MINUTE, 60) === false)
{
	return Services::response()->setStatusCode(429);
}

Runtime: 10 seconds
Rate of 81 responses per second
- Code 200 was responded 12 times.
- Code 429 was responded 797 times.

Runtime: 60 seconds
Rate of 80.9 responses per second
- Code 200 was responded 62 times.
- Code 429 was responded 4791 times.

[natanfelles]

Of course, I believe that my test script must be only while (time() < $stop_at).

Then:

Runtime: 1 seconds
Rate of 73 responses per second
- Code 200 was responded 2 times.
- Code 429 was responded 70 times.

Runtime: 10 seconds
Rate of 75 responses per second
- Code 200 was responded 11 times.
- Code 429 was responded 738 times.

Runtime: 60 seconds
Rate of 77.5 responses per second
- Code 200 was responded 61 times.
- Code 429 was responded 4588 times.

Before all tests was run rm -rf writable/cache/*.

[natanfelles]

...and the $requests = 0; // The requests counter.

[lonnieezell]

Hmm.. it sounds like it's partially working then. But I wonder if it's filling itself back up faster than it should....