Wednesday, 2012-07-25

*** Joins: giallu (~giallu@fedora/giallu)00:32
*** Joins: Paul_46 (~IceChat09@cpc1-enfi15-2-0-cust580.hari.cable.virginmedia.com)02:20
*** Quits: Paul_46 (~IceChat09@cpc1-enfi15-2-0-cust580.hari.cable.virginmedia.com) (Ping timeout: 246 seconds)03:06
*** Quits: giallu (~giallu@fedora/giallu) (Read error: Operation timed out)03:07
*** Joins: Paul_46 (~IceChat09@cpc1-enfi15-2-0-cust580.hari.cable.virginmedia.com)03:43
*** Joins: giallu (~giallu@fedora/giallu)07:17
*** Quits: giallu (~giallu@fedora/giallu) (Ping timeout: 276 seconds)10:21
*** Joins: BD- (BD-@78-22-91-42.access.telenet.be)11:00
*** Quits: BD- (BD-@78-22-91-42.access.telenet.be) ()11:22
*** Joins: BD- (BD-@78-22-91-42.access.telenet.be)11:25
*** Quits: BD- (BD-@78-22-91-42.access.telenet.be) (Client Quit)11:27
*** Joins: BD- (BD-@78-22-91-42.access.telenet.be)11:27
*** Quits: BD- (BD-@78-22-91-42.access.telenet.be) ()11:56
*** Joins: BD- (BD-@78-22-91-42.access.telenet.be)11:56
*** Quits: BD- (BD-@78-22-91-42.access.telenet.be) (Client Quit)11:59
*** Joins: BD- (BD-@78-22-91-42.access.telenet.be)12:01
*** Joins: giallu (~giallu@fedora/giallu)13:19
*** Joins: dregad (b2c5e92b@gateway/web/freenode/ip.178.197.233.43)14:19
*** Quits: BD- (BD-@78-22-91-42.access.telenet.be) ()14:41
dregadjreese - you there ?14:42
jreeseyessir14:42
dregadhey man14:43
dregadhow are things ?14:43
jreesepretty good, how are you?14:43
dregadcan't complain - on vacation :)14:43
dregadspent the day at Europa-Park (kinda disneyland) with the family14:44
dregadthey're sleeping now :-)14:44
dregadi've been getting daily alerts from linode regarding disk io rate lately14:45
jreeseyeah, I think they're being triggered by the daily backups14:46
dregadwas looking into it, and looking at graphs, it seems that things have been getting worse (IO steadily increasing over past 30 days, see https://manager.linode.com/linodes/graphs/mantisbt_org?date=monthly14:47
dregadyou're right they are triggered by the backup14:47
dregadbut on a "normal" load (i.e. looking at monthly graphs for earlier in the year), the backups did not trigger any alert14:48
dregadif it were memory i would say there's a memory leak somewhere, but since it's IO, it must be a file getting bigger and bigger14:48
dregadI don't think the DB's been increasing in size, so it's not the backups themselves using more bandwidth14:49
dregadthe activity on the tracker has been quite low, and the forums not more busy than usual either14:50
jreesemight be log files14:50
jreeseor database sizes growing14:50
dregadI already ruled out the latter14:51
jreeseOk14:51
dregadlog files make sense, but in /var/log I don't see anything unusual14:51
Paul_46dregad: lo14:51
jreesewell, /var/log/apache/ and mail.log* will be continuously growing with traffic to the site and tracker14:52
Paul_46we do a daily backup ? :PPP14:52
* dregad MOOs Paul_4614:52
jreesenot sure if the size is what triggers the extra IO during a backup14:52
* dregad kicks Paul_4614:52
jreesePaul_46: Victor enabled the Linode-native system backups14:52
dregadI was wondering if it could be the IRC logger14:53
dregadLooking at http://www.mantisbt.org/irclogs/ there is a HUGE log file 'cron.log'14:53
dregadnearly 300 megs atm14:54
Paul_46downloading it14:54
Paul_46shows lots of 'cp' statements14:54
Paul_46:)14:54
dregadyep14:54
Paul_46bw is free thoiugh right?14:54
dregaddunno14:54
dregadthink there's a quota14:55
dregadjreese as I'm not too familiar with how you've setup the IRC logs (if I remember correctly what you told me before, your own server is pushing the files to mantisbt.org, right ?)14:57
dregadmaybe this log file should not be copied over, or if yes, then maybe it could be truncated (or rotated)14:57
dregadwdyt?14:58
Paul_46could just copy the logs from my server14:58
Paul_46;p14:58
Paul_46hmm, that files still downloading14:59
dregadlol@Paul_4615:00
jreesedregad: yeah, that file should probably get truncated or ignored or something, didn't realize it was growing that much15:00
dregadPaul_46 - actually I think the right way to do things would be to setup a bot directly on mantisbt.org15:01
Paul_46I think we should stop advertising gambling sites on mantisbt.org15:03
* dregad couldn't agree more15:03
Paul_46haha15:04
Paul_46that's funny15:04
Paul_46downloading that cron.log crashed IE15:04
Paul_46so it's started again15:04
dregadserves you right for using that POS browser15:04
Paul_46that's like 500MB of download15:04
dregaddon't bother downloading it15:04
Paul_46or something15:04
Paul_46I wanna know if it crashes again15:04
Paul_46i'll attach a debugger and check it's not av plugin15:05
dregadyou like beating on a dead horse don't you15:05
Paul_46i code on mantis ;p15:05
dregadlol15:05
jreesezing15:05
dregadjreese considering that the logs are being refreshed every 10 minutes iirc, that's 300MB * 6 per hour * 24 = 42 GB...15:06
dregadwhat do you mean 'zing', you zapped the file ?15:06
Paul_46http://www.fosspatents.com/2012/07/apple-seeks-25-billion-in-damages-from.html15:07
jreeseno, "zing" as in "good insult"15:07
Paul_46apple want 2.5b$ from samsung15:07
dregadso what else is new, these 2 have been going at each other for months now15:08
jreesedregad: considering that it uses rsync, it's not quite 42GB ;)15:08
dregadwell i don't know what it uses, so :P15:09
jreesethe script is synced alongside the results :P15:09
* dregad should learn Python some day15:10
dregadand15:11
jreeseit's not that difficult :P15:11
* dregad should also have checked the shell script file next to the python file too ;)15:11
*** Parts: orlok (~orlock@182.160.128.150) ()15:14
dregadanyway, jreese could you try truncating that log, to see if the IO goes down15:14
jreesesure15:14
dregadcheers15:14
Paul_46anyway, it's kinda amusing no one's ever given me details since we moved server15:15
Paul_46been over a year now15:15
jreeseyou haven't asked15:15
jreesedregad: crontab updated to truncate the log on every run, and also only output stderr, rather than stderr + stdout15:19
dregadthanks john15:20
jreesenp15:20
dregadwe'll see if I was right or not at the next backup I guess15:20
Paul_46btw15:21
Paul_46not sure what to do about the dude with he's security issues15:21
Paul_46Globus Healthy & Safe Medifast Diet We like this mommy blog a lot.15:21
Paul_46and wtf15:21
jreesewe were planning to do something? :P15:21
* Paul_46 shrugs15:21
dregadwhich dude are you talking about here15:22
Paul_46anyway15:24
Paul_46i was saying last night or well looking last night15:24
Paul_46and trying to explain to bd- where things were15:24
Paul_46he asked about why we have a next, master, master-1.2 and some other stuff15:25
dregadLOL15:25
Paul_46so I through in the master-2.x and tried to do some history15:25
dregadI don't think anybody knows ;)15:25
jreesedregad: he sent a laundry list of vulnerabilities that were either issues with our upstream libraries, or unintelligible mess of complaints15:25
Paul_46yea15:25
Paul_46he's refined it somewhat15:25
dregadah yes, that15:25
Paul_46but still semi similar15:25
dregadI remember now15:25
Paul_46but yea15:27
Paul_46not quite sure what to do with issues in upstream lib's we getting rid of15:27
Paul_46at a point when it's a mess15:28
Paul_46dregad: i looked last night at github's stats15:28
Paul_46whilst talking to bd15:28
*** Quits: giallu (~giallu@fedora/giallu) (Ping timeout: 276 seconds)15:28
dregadwhat about nothing15:28
Paul_46specifically: https://github.com/mantisbt/mantisbt/graphs/contributors15:28
Paul_46if I go back to Jan 2009 to now15:29
Paul_46active commiters: dhx, me, jreese, you, rombert, siebrand, victor [ based on >100 commits in 3 years]15:29
Paul_46if I come forward a year and use 60 i.e. 30 a year15:30
dregadyes, and if you look at past 6 months, it's 95% me and rombert15:30
Paul_46active commiters: you, dhx, rombert, siebrand, jreese15:30
dregadand 99.999% of what rombert does is soap ;)15:30
Paul_46siebrand can be excluded15:30
Paul_46jreese can be excluded15:30
dregadjreese is mostly the releases15:30
dregaddhx is a few security patches mostly15:31
dregadand 'next'15:31
Paul_46so since Jan 2010, you have 181 commits, rombert is 11715:31
Paul_46(dhx is 146)15:31
Paul_46interestingly, if I do from jan 2010 on my branch idhx has done 268 commits [not sure how that works], and i've done 21115:33
dregadI never actually looked at these graphs, they're nice... can you filter the contributors by date ?15:33
Paul_46yea15:34
Paul_46if you drag on graph date range15:34
jreesedregad: the graphs have had significant improvements in the last few months; they weren't even half this good before then15:35
Paul_46yep15:35
Paul_46now i'm unsure where i stand on mantis atm15:36
Paul_46as I kinda enjoy it15:36
Paul_46but as far as i'm concerned, 1.2 and 1.3 are dead15:36
dregadreally cool these graphs, like them15:36
dregadthese github dudes are good15:36
jreeseyep15:37
dregadPaul_46 - the problem, as we have discussed several times in the past, is the lack of vision on 'what's next' (no pun on branch naming intended)15:37
jreesedregad: that's what I said yesterday :P15:37
dregadyou do your thing, dhx does his15:38
dregadand I maintain 1.2.x because I use it and that's where issues arise15:38
jreese"<jreese> [15:23] one thing we've always lacked was a clear set of goals for a release; eg, either a hard deadline or a hard set of features to release"15:38
Paul_46yep15:38
Paul_46dregad: yes, but we've had new features go into 1.215:40
* dregad pleads guilty15:41
dregadbut your Honor, it's because 1.3/2.x/next is so far away - the People *need* features15:41
Paul_46so there's new features in 1.2 that personally I think we should deprecate for 1.3 and drop15:41
Paul_46(in some cases)15:41
Paul_46or at least change how they work15:41
Paul_46and that lack of a plan/vision isn't helping15:42
Paul_46hey, one of the things i'd like to add is a free open source mobile interface15:42
Paul_46apart from whether that would kill off mantistouch or not15:43
Paul_46i'd also add a different mobile api for that to what got added between 1.2.8 and 1.2.9 or whatever it was15:43
Paul_46the *easy* way for dealing with that would be to set up mantisbt2.org15:43
Paul_46to differeniate between the two versions of the api's15:44
Paul_46or not bother15:45
jreeseuser confusion ftw15:46
Paul_46jreese: well given that we'd pretty much agreed for mantis 2.0 we'd probably want to drop the current soap api15:47
Paul_46might be less confusing then releasing a 1.315:47
dregadI don't think it's a good idea to "officially" fork by setting up a separate site, etc15:48
dregadit would not only even further dilute and shrink the team, but also as jreese mention, cause user confusion and likely very few would follow to the new version as it essentially breaks the existing15:49
Paul_46well, we have that problem anyway15:50
Paul_46as 1.3/2.0/next [whichever you want to call it] is likely to change db api in a way that woudl require some level of plugin updates15:50
Paul_46and i'd like to look at whether it's possible to tidy up internal structure to make it easier to do things like mobile versions and soap api's15:51
Paul_46and xmlrpc and json etc if people want15:51
Paul_46dregad: and in terms of that, lets clarify a bit15:54
Paul_46November 2009 we had a 'web server api v2 brainstorming' thing on google wave15:54
Paul_46and it listed some goals15:54
Paul_46Goal Number 1: Authentication - don't use the users password15:55
Paul_46Goal 2: Authorization - ability to limit access to an application15:55
Paul_46Goal 3: performance and scability - efficient memory use, throttling, batching15:56
Paul_46Goal 4: Versioning - server side api version, clients provide schema they support, server reponses in version expected by client15:56
Paul_46list went on with another 7-8 things15:57
Paul_46now personally, I like goal 1, and i think it's important15:57
* dregad wasn't around back then15:57
Paul_46user should log into mantis15:57
Paul_46generate an "api key(s)"15:57
jreesedregad: yeah, I think that was also before dhx15:57
Paul_46webservice should then use that key15:57
Paul_46jreese: dont think so15:58
Paul_46he's in the discussion actually15:58
jreeseoh, ok15:58
Paul_46anyway, given the number of password leaks etc15:58
Paul_46would you agree goal 1 is probably quite important and reasonable a goal? :)15:58
Paul_46if no, i'd be interested to hear your argument on that one15:58
jreesePaul_46: it might be easier to just implement an OAuth provider :P15:59
Paul_46if yes, I suspect implementing that goal would break the current soap api15:59
Paul_46as auth method would probably change from user,pass to authkey if nothing else15:59
jreesenow that there are some good standards around public API authentication and authorization15:59
Paul_46nod15:59
* dregad agrees16:00
Paul_46on the mantis-2 stuff i'd done, i'd basically gone rm -rf soap16:00
Paul_46not because i necessarily want it gone16:00
Paul_46but more, I want to be able to think about mantis and not worry about stuff I can't easily test16:00
dregadquite frankly Paul, I think the only problem with your branch is that it has diverged too much from current 1.2.x16:01
Paul_46and work on seeing if it's possible to modernise the legacy core16:01
dregadmostly due to my work, it's true16:01
Paul_46but that's also kinda the aim16:01
Paul_46there's a lot in 1.2 that was done for a reason16:01
dregadthe advantage of 'next' is that dhx has frequently rebased it on top of master16:01
Paul_46in some cases a good reason16:01
Paul_46i'm pretty sure that's got to the point where even dhx isn't keen on the file system layout in it though16:02
dregadanyway, switching to your branch would cause either16:02
dregada) loss of hundreds of commits you have not ported16:02
Paul_46i'm not overly concerned about that16:02
Paul_46as, from our stats earlier16:02
dregador b) generate a lot of work to recode stuff16:02
Paul_461:30.09] <Paul_46> if I come forward a year and use 60 i.e. 30 a year16:03
Paul_46[11:30.12] <dregad> yes, and if you look at past 6 months, it's 95% me and rombert16:03
Paul_46[11:30.29] <Paul_46> active commiters: you, dhx, rombert, siebrand, jreese16:03
Paul_46[11:30.37] <dregad> and 99.999% of what rombert does is soap ;)16:03
Paul_46so twisting those numbers a little16:03
Paul_4695% of the stuff in mantis is you+rombert and 100% of rombert's is soap [which needs a rewrite]16:04
Paul_46so if we assuume that you and rombert are roughly  50-5016:04
Paul_46well probably 60-4016:04
dregadhttps://github.com/mantisbt/mantisbt/graphs/contributors?from=2012-01-01&to=2012-07-22&type=c16:04
Paul_46that's basically 35% of stuff in last 6 months that can be ignored16:04
dregadmore like 80%+ me16:04
Paul_46and your stuff is a mix of stuff that is minor16:06
Paul_46and stuff that would require porting16:06
Paul_46some of which is already ported, some is no longer relevant16:06
Paul_46in any case, it's 100 commits16:06
* dregad finished reading yesterday's IRC log transcript - nice to know the story behind the nick :P~~16:06
Paul_46heh16:07
Paul_46so 100 commits to 'review' I dont see as a major issue16:07
dregadwell actually you should go back to the date of your fork16:07
dregadI just picked 01-jan-1216:07
Paul_46yea16:07
Paul_46but in any case16:07
dregadIn my opinion, you should work with dhx to merge your stuff with 'next'16:08
dregadthen we can kill 1.316:08
dregadannounce 2.016:08
dregadand be done with i16:08
dregadit16:08
Paul_46i've not seen dhx for a while16:08
dregadme neither, but I have not been online much lately16:09
Paul_46and I dont like the current filesystem layout of 'next'16:09
dregadbah16:09
dregadyou'll get used to it16:09
dregadjust like I'll have to get used to the new db syntax, etc16:09
Paul_46april 201116:10
Paul_462011-04-1116:10
dregadwell that's way over 250 commits16:11
Paul_46i dont see that as an issue though16:11
dregadnot counting rombert's btw16:11
Paul_46as a bunch are soap, translation16:11
Paul_46and some ported in already16:12
Paul_46in any case, 250 commits isn't that many16:12
dregadstill, next has most of them already16:12
Paul_46yea, but there's no way you'd be merging this stuff in16:12
Paul_46easily16:12
dregadso merging your branch with next might be less effort than reviewing / porting 250 commits16:13
dregadeven if manual merge16:13
dregadlisten i gotta go - it's getting late16:14
dregadwe should continue this conversation sometime16:14
dregadpreferably including dhx too16:14
dregad@jreese - looks like truncating the log did the trick... https://manager.linode.com/linodes/graphs/mantisbt_org?date=daily16:15
Paul_46225 commits in this branch16:15
dregadhow much of it already included in 'next'16:15
jreesedregad: excellent16:15
Paul_46dregad: not sure16:16
Paul_46in any case, it's gonna be a pain either way16:16
Paul_46to sort out16:16
dregadfor sure16:16
dregadbut if we go 2.0 --> next, we keep the exceptions stuff, which I think is great16:17
Paul_46we're doing exceptions either way16:17
Paul_46the difference is i've not implemented handlers for exceptions16:17
dregadwhereas if we hand-pick commits from 1.3 to your 2.0, we have to also redo the exceptions16:17
Paul_46?16:17
dregadI mean, re-code what dhx has done in next16:18
dregadinto your branch16:18
Paul_46and i'm currently throwing exceptions as:16:18
Paul_46throw new MantisBT\Exception\Category_Duplicate();16:18
jreesegad I hate PHP's namespacing choice16:18
Paul_46whereas dhx is doing:16:19
Paul_46use MantisBT\Exception\Security\CSPRNGNotAvailable;16:19
Paul_46and then throw new MasterSaltInvalid();16:19
dregadin other words, your saying you don't like dhx's way ;)16:19
Paul_46personally, if it's possible to do throw new MantisBTException\Duplicate16:19
Paul_46nah16:19
Paul_46not that actually16:19
Paul_46i'd say they are fairly similar16:20
Paul_46i'm not sure what people commonly do16:20
Paul_46additionally i'd not bothered to make exception classes for each exception16:20
Paul_46on the basis I didn't see the point of doing that until we worked out what to throw16:20
dregadlook i really have to go - it's 1.30 am and the baby's gonna give me a wake-up call in about 5 hrs :-/16:21
Paul_46and as a minor statement to back that up:16:21
Paul_46https://github.com/mantisbt/mantisbt/blob/next/application/MantisBT/Exception/Filter/FilterTooOldToUpgrade.php16:21
Paul_46$this->responseCode = 500;16:21
Paul_46personally, given exception could get thrown by a cmdline cron job16:21
Paul_46i'm not sure it makes sense to set a 'responseCode' in the exception16:21
Paul_46$this->type = MANTIS_FATAL; sure16:22
Paul_46or something16:22
Paul_46nn ;)16:22
dregadgood night16:22
*** Quits: dregad (b2c5e92b@gateway/web/freenode/ip.178.197.233.43) (Quit: Page closed)16:22
*** Quits: Paul_46 (~IceChat09@cpc1-enfi15-2-0-cust580.hari.cable.virginmedia.com) (Quit: ASCII a stupid question, get a stupid ANSI!)16:50
*** Quits: sdfjkljkdfsljkl (~sdfjkljkd@static.96.23.63.178.clients.your-server.de) (Remote host closed the connection)17:00
*** Joins: sdfjkljkdfsljkl (~sdfjkljkd@static.96.23.63.178.clients.your-server.de)17:00
*** Joins: giallu (~giallu@fedora/giallu)23:47

Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!