Subject: Re: logging to sqlite db? - msg#00156

Previous Message by Date: click to view message preview

Re: logging to sqlite db?

On 26/07/05, aaron <aaronyy@xxxxxxxxx> wrote: > --- Aled Gest <himselfe@xxxxxxxxx> wrote: > > Which is what my original statement suggests: Output > > the log in a DB > > format. > > > > It is a very blunt statement, IMHO. Relation database > cannot be simplified as a format. > > Indeed it is a very blunt statement. But it was intentionaly oversimplified. I am simply trying to provide constructive ideas on how DB logging could be done in the most efficient way. My statement is no more blunt than people simply stating it's a bad idea and should not be done. I dont quite understand where the hostility towards discussing the posibility in a constructive manner comes from. It would be far simpler to express why the original idea is flawed and either how it can be improved or suggest an alternative more efficient method of acheaving the objective should sombody be determined to use this method of logging.

Next Message by Date: click to view message preview

Re: logging to sqlite db?

Aled Gest wrote: On 26/07/05, aaron <aaronyy@xxxxxxxxx> wrote: --- Aled Gest <himselfe@xxxxxxxxx> wrote: Which is what my original statement suggests: Output the log in a DB format. It is a very blunt statement, IMHO. Relation database cannot be simplified as a format. Indeed it is a very blunt statement. But it was intentionaly oversimplified. I am simply trying to provide constructive ideas on how DB logging could be done in the most efficient way. My statement is no more blunt than people simply stating it's a bad idea and should not be done. Aled, Re-read the thread - most of the people who've said it is a "bad idea" have given reasons why. > I dont quite understand where the hostility towards discussing the posibility in a constructive manner comes from. Which thread are you reading? As I read it, you've suggested an idea and a few people have taken the time to respond and suggest why that is not a good idea. You seem to be getting heated about this, but I'm not sure why. It would be far simpler to express why the original idea is flawed and either how it can be improved or suggest an alternative more efficient method of acheaving the objective should sombody be determined to use this method of logging. Both of these have been done. Generally speaking, the optimal approach to doing this is to write the logs to disk in some sort of parseable text format and then post-process the text files in whatever way you see fit, e.g. load them into some sort of database. R. -- http://robinbowes.com If a man speaks in a forest, and his wife's not there, is he still wrong?

Previous Message by Thread: click to view message preview

Re: logging to sqlite db?

> I can see the benefit of logging to a process (though a pipe). The process > can do whatever it want's to put log-entry's into a DB, whatever.. Well as i've mentioned, how the process of loging to a DB is performed is open for inspiration. The original request was simply a way of being able to log to an SQLite DB. > I don't see the benefit of letting lighty do this .. it's extra code, > extra things which can go wrong (to wait for the db when logging means > the entire webserver waits for the db.. ). Like i've already mentioned, lighty doesnt need to access the DB to be able to output data the DB can access, i have already voiced my opinion on how making lighty itself use SQL is a bad idea as it adds redundant overhead. Also, having it implemented as a module means it generaly has no effect on the rest of lighty's code (other than code that depends on that specific module). It also means people who dont want to use DB logging don't have to load it, people can choose to load mod_accesslog or people can choose to load mod_accesslogdb, it's entierly up to the end user. And as it states in lighty's configuration documentation, you should not load modules you don't need or use, if you dont have the module loaded it's not going to cause you any headaches or overhead nomater how badly it performs. And having a module that outputs loging data in a format that can be picked up by DBs does not mean you have to implement (for example) SQL in lighty itself, it just means you have a module that provides an interface for those DBs to access the data. And having lighty queueing data ready for the DB to process does not mean lighty has to wait for the DB to process it. > Not really. In my opinion logging to a (any) DB is harmfull, and even more > harmful if it's badly implemented. As many people have stated, you believe loging to any DB is harmfull, you have still failed to explain HOW it is harmfull (not concerning the issues already covered above). > It would make the whole server a little bit more bloated. (and more difficult > to compile, maintain, secure/chroot, etc) Only if you decide to use that module in the first place. Perhaps you have misunderstood the meaning of modular design. If you dont enable (puerly for example) --with-sqlite at the configure stage of the build process, then mod_accesslogsqlite wont be included for compilation. If you're using a pre-compiled binary distro of lighty, that has mod_accesslogsqlite included, but you dont enable it in lighttpd.conf, it wont be loaded and lighty wont even know it's there. The great thing about the modular design of lighty is the fact that people can include the features they need as modules and aren't forced to load the bits they dont need or want. As for maintaining the code, the maintanence of one module doesn not have to concern the person dealing with the core of the program. If Jan wants to or doesnt want to implement this as a module that's up to him. If anybody else on this list likes the idea and wants to code the module to do it they're free to do it. Remember this is an open discussion list where anybody is free to request a feature, and anybody is free to contribute to said feature. It is not a "Jan implement this feature in to the core of lighty so everybody's forced to use it" list. > > It's called feature creep > > > espcialy if it's an optional module, where you have a > > choice between standard logging, syslog, or a DB logger. Obviously > > it's up to Jan if he wants to implement this as an official feature in > > lighty or not. But anybody is free to develop any module they believe > > is needed. > > > ---end quoted text--- > >

Next Message by Thread: click to view message preview

Re: logging to sqlite db?

> I think you didn't read the first email I wrote. There > are overhead in term of both implementation and > operation. > > Using a DB engine inside lighttpd which is not a good > idea because of the single threaded design. > Writing a separate DB logging engine is a redundant > effort. No i did read your first e-mail, and yes you do explain why implementing SQLite directly into lighty is a bad idea, and for your explanation i have absolutly no argument. The comment you quoted was in reply to paulv's e-mail. And in my very first e-mail i already expressed my own reasons as to why implementing any form of SQL interface for logging directly in lighty would be a bad idea. I have not intentionaly recomended that any DB engine be used directly in lighty. I am simply trying to discuss the idea of lighty having a module that supports an external database's ability to provide access to the logged data. I personaly have no need for this feature, but if you would read my discussion from the start i am simply trying to provide constructive comentry on the issue.