Enhance the access log format

[craigminihan]

Currently the access log outputs the following format:

/_utils/index.html METHOD: GET
/_utils/style/layout.css?0.11.0 METHOD: GET
/_utils/script/jquery.js METHOD: GET
/_utils/script/jquery.dialog.js METHOD: GET
/_utils/script/json2.js METHOD: GET

It would be useful if a standard format was used, the most obvious being: http://en.wikipedia.org/wiki/Common_Log_Format. A more adventurous alternate would be the MS IIS extended log file format: https://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/bea506fd-38bc-4850-a4fb-e3a0379d321f.mspx?mfr=true, this is a customizable format which has the interesting time-taken field which is critical for performance analysis of REST APIs.

[etr]

I think that using a standard logging format is more than just needed when building a web server.

The library itself though is not a web server, but a tool to build one. The responsibility of the library (for how it is designed) toward logging it's to provide a support to plug a logging system.

Right now, when constructing a webserver object, it is possible to pass handlers for access_log and error_log.

The log lines you are attaching to your comment come from a mis-implementation of the logging in the library (it is a refuse from the early stages of development when everything was a single big method).

I think the right thing to do, would be to improve the interface of access_log and error_log so that (at least) a standard log format could be used.

Said so, I do not think that building a logger is totally out of board - but I think it should be isolated from the rest of the library and plugged using the interface (maybe shipped with the library as a default replaceable logger).

[craigminihan]

@etr I've looked at webserver::answer_to_connection where access_log is called from. It looks to me that other than url/method/version/etc passed as params to answer there isn't any other useful information available. It would be nice to get MHD_Connection* connection but this is an internal decl in libmicrohttpd which is a shame.

[fechantillac-antidot]

Interested in this too. Migrating from fastcgi to libhttpserver, given the lack of information in the current access log we plan to keep using apache as a proxy. What would be useful would be to get some informations about the response in the access log callback. At least the status code. Size and response time would be quite useful as well.

Is it something you may be addressing in the current year ?

FYI we are using spdlog as a logging facility.

[etr]

One of the main feature I want to release this year is a system of interceptors at global ws level (pre-routing, post-routing) and at resource level (pre-handling, post-handling). The initial intent was to tackle cases like ( #102 ). That should allow for any level of elasticity because those interceptor would get the full http_request in input (so from there you could log anything really).

I think I will keep the access_log method around, but that is likely going to be limited because it is called before the request is even parsed (despite having the benefit that it is very hard for it to ever miss anything). I might enrich that with a better set of data, but as said, it would be limited there.

For your case, I think that the interceptors should help the most.

[EverydayDiesel]

I would also like to see this functionality. I am still trying to figure out how to get the ip address of the requester

[etr]

@EverydayDiesel you should already be able to access the ip address of the requester using the get_requestor method from the http_request; see: https://github.com/etr/libhttpserver#parsing-requests

[EverydayDiesel]

@EverydayDiesel you should already be able to access the ip address of the requester using the get_requestor method from the http_request; see: https://github.com/etr/libhttpserver#parsing-requests

Hello so sorry to bother you. Is it possible to get the user agent ie something like this?
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

I tried all of these but never got it

    std::cout << "IP Address: " << req.get_requestor() << std::endl;
    std::cout << "Get Version: " << req.get_version() << std::endl;
    std::cout << "Requestor: " << req.get_requestor() << std::endl;
    std::cout << "Requestor Port: " << req.get_requestor_port() << std::endl;
    std::cout << "Version: " << req.get_version() << std::endl;
    std::cout << "Content: " << req.get_content() << std::endl;

[etr]

This is getting off topic from what this thread was about, so if you need further info on how to use the APIs, please open a separate one or use gitter.

The user agent is a header, not a detail of the connection. As such, you can access it through the headers within the request, provided that your client is passing them. The get_header and get_headers methods in request (https://github.com/etr/libhttpserver#parsing-requests) are what you are looking for.

[Lotharyx]

One of the main feature I want to release this year [2020] is a system of interceptors at global ws level (pre-routing, post-routing) and at resource level (pre-handling, post-handling).

Did this get released in the past couple years? I'm (seemingly) stuck having all of my resource classes call a common logging method (which takes the request object as an argument) to get the log info formatted the way I want. Is there a better way?

[etr]

Resolved in libhttpserver 2.0.0. The new lifecycle-hook system adds a response_sent hook that fires after each response with the HTTP status code, bytes queued, and elapsed time — enough to emit Common Log Format including the time-taken field you mentioned. There's a ready-made example at examples/clf_access_log.cpp, and the hooks are documented in the README's Lifecycle Hooks section.

Closing as resolved by 2.0.0.