[01:04:57] DracoBlue (~Adium@dslb-088-075-066-254.pools.arcor-ip.net) joined #rest. [01:05:07] DracoBlue (~Adium@dslb-088-075-066-254.pools.arcor-ip.net) left #rest. [01:06:11] grove (~grove@aggw006.cappelendamm.no) left irc: Quit: grove [01:08:44] grove (~grove@aggw006.cappelendamm.no) joined #rest. [02:34:01] grove_ (~grove@aggw006.cappelendamm.no) joined #rest. [02:36:52] grove (~grove@aggw006.cappelendamm.no) left irc: Ping timeout: 252 seconds [02:36:52] Nick change: grove_ -> grove [03:20:24] mr_yall (~mryall@scifo.mexico86.co.uk) joined #rest. [03:21:30] Nick change: mr_yall -> mryall [03:26:16] Nick change: mryall -> mr_yall [03:37:26] mikekelly (mikek@s3x0r.biz) left irc: Read error: Connection reset by peer [03:37:33] mikekelly (mikek@64.32.20.15) joined #rest. [04:38:33] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) joined #rest. [05:04:20] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [05:44:01] Gracenotes_ (~person@wikipedia/Gracenotes) joined #rest. [06:00:47] Nick change: Gracenotes_ -> Gracenotes [06:13:33] DracoBlue (~Adium@dslb-088-075-066-254.pools.arcor-ip.net) joined #rest. [06:13:45] DracoBlue (~Adium@dslb-088-075-066-254.pools.arcor-ip.net) left #rest. [06:21:59] grove (~grove@aggw006.cappelendamm.no) left irc: Quit: grove [06:29:19] Gracenotes (~person@wikipedia/Gracenotes) got netsplit. [06:29:19] KarlHungus (~relax@unaffiliated/adj) got netsplit. [06:31:38] Gracenotes (~person@wikipedia/Gracenotes) returned to #rest. [06:31:38] KarlHungus (~relax@unaffiliated/adj) returned to #rest. [06:47:20] mamund: that is an awesome example of what I think should be happening w/ app design [06:47:27] yes yes yes. [06:47:27] :) [06:48:42] darrelmiller: thanks for replying to that guy [06:50:27] mikekelly: np. :-) [07:06:03] sbanwart (~sbanwart@66.181.75.251) joined #rest. [07:11:34] sbanwart (~sbanwart@66.181.75.251) left irc: Ping timeout: 255 seconds [07:24:42] mephju (~mephju@dslb-188-103-207-215.pools.arcor-ip.net) joined #rest. [07:29:37] grove (~grove@193.201.9.46.customer.cdi.no) joined #rest. [07:30:27] Action: mamund has arrived [07:31:44] Action: steveklabnik waves at mamund [07:31:57] steveklabnik: scanning the maze solver right now [07:32:27] i'm going to do some more with it today, i want to show the 'from nothing -> procedural' part too [07:32:32] should involve some fun git-fu [07:32:34] cool [07:32:37] adding commits at the beginning [07:32:51] the README has links to the tags for both versions [07:32:57] ahh [07:33:25] LOL, love the 'commit as you go' notes [07:33:35] grove (~grove@193.201.9.46.customer.cdi.no) left irc: Read error: Connection reset by peer [07:33:41] i've had the revelation a couple times in the past, too. [07:33:49] grove (~grove@193.201.9.46.customer.cdi.no) joined #rest. [07:34:14] I'm actually going to expand on them, too [07:34:22] very instructive. [07:34:22] it [07:34:23] 'my new blog post is a git repo' [07:34:37] it's more work to do that (track commits, comment as you go, etc.) [07:34:42] but i think it does a great service [07:35:00] it can expose a _methodology_ for building in this way [07:35:06] yeah. [07:35:09] a road map that others can use as a guide [07:35:14] good stuff [07:35:41] Action: steveklabnik bows [07:35:44] :) [07:36:51] astute readers will notice that the procedural version is one file and like 60 lines, and the OO version is about 6 files and 120ishlines. [07:37:07] ... not that it should be a surprise, but it's something... [07:37:08] yep [07:37:47] building in abstractions to support flexibility, etc. always has a "cost" [07:37:52] Action: steveklabnik nods [07:38:28] fuck flexibility [07:38:41] write everything in callbacks [07:38:45] it's web scale [07:38:54] hahah [07:39:01] mikekelly: callbacks as anonymous functions! [07:39:17] I think the thing that's extra awesome about node.js and javascript [07:39:36] is that most people use anonymouse functions without realising what that actually means in terms of performance cost [07:39:51] meh [07:39:54] you just need a good VM [07:40:21] ont at WEB SCALE [07:40:23] not^ [07:40:24] well, that's just one of the many "i have no idea of the implicatiions of that code snippet i pull from that aw3sOmE kEwl website" [07:41:00] I dunno, I'm just really beardy about my js and optimise out all the callbacks first time [07:41:21] I talk about performance because it makes me feel better that my code reads like poop [07:41:28] yeah, mamund, that's what i was trying NOT to say yesterday [07:41:37] ?? [07:41:42] 'node is full of designers that want to try to code, and they all suck at it' [07:41:43] ;) [07:41:46] ahhh [07:41:49] LOL [07:41:54] oh damn [07:42:11] Action: mikekelly copy pastes into #node.js [07:42:22] heeeeeee! [07:42:23] lol [07:42:32] steveklabnik: and rails doesn't have any of those types, eh? ;-) [07:42:44] oh, it has lots of noobs, dont get me wrong [07:42:52] but the noobs aren't the ones who are yelling the loudest [07:42:59] 'oh my god, I only have to know one language now!' [07:43:02] the m$ ecosystem is the shittest I have ever seen [07:43:30] I would rather hack at someones hack in php than work on any of the horrible stacks you get in m$ systems [07:43:32] they are the equivalent of Microsoft's drag and drop, code gen, wizard heads [07:43:41] fucking [07:43:45] IIS configuration [07:43:47] mikekelly: :-P [07:43:57] was created an implemented by people on HEAVY drugs [07:43:59] ASP.NET has a similar problem. only it's green-screen CICS guys from 1990 writing 'web code' ("cuz it's all drag-n-drop now!") [07:44:35] I spent at whole day pissing around with IIS configuration [07:44:40] a whole day. [07:45:17] all I was trying to do was configure it to catch error responses and serve a static page [07:45:31] retarded. [07:45:43] :) [07:45:45] heh. [07:45:55] this is why i dont touch M$ [07:45:55] my microsoft career lasted < 6 weeks. [07:45:57] other than ethics [07:46:22] still [07:46:48] at least I haven't had to go back to php [07:46:55] now THAT is a shit language [07:47:20] :) [07:47:31] they haven't even picked up on the wsgi/rack thing yet [07:47:32] tell that to dzuelke [07:48:02] someone's implemented rack for php but nobody builds on it [07:48:41] astonishing. [07:48:51] heh [07:52:30] unsuccesful troll is unsuccessful [07:52:34] :( [08:19:32] gchristensen (~gchristen@unaffiliated/grahamc) left irc: Quit: Leaving... [08:21:28] bradley-holt (~bradley-h@65-183-135-35-dhcp.burlingtontelecom.net) joined #rest. [08:33:41] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [08:37:01] mamund: any clue where we are at after that google+ JSON linking extravaganza ? [08:37:45] not a clue [08:38:18] think it the thread expanded to a cold death [08:38:54] lol [08:42:25] http://img.photobucket.com/albums/v93/Pandareus/success_baby1.jpg [08:42:49] :) [08:44:28] mamund: I'm also confused about Erik's comments about specific types and 'processing models' and using css as an example [08:44:48] well, yeah.... [08:45:08] do you understand the point he's making ? [08:45:23] CSS *is* a read-only hypermedia type, but not sure it's in the same "class" as the ones we were talking about (JSON, etc.) [08:45:35] not sure of his point (any more) [08:45:55] plus the whole thing didn't really help his point [08:46:03] maybe that curly-braced, rules-engines are viable hypermedia formats? [08:46:33] well, i think he was trying to indicate the useragents "know" (from that rel) how to "process" a message [08:46:36] just guessing, tho [08:46:42] i didn't pursue it w/ him [08:46:44] yeah I was trying to find a way to respond to it but it got long winded and I deleted the draft [08:46:51] LOL [08:47:26] well afaik he was contesting that rels dont' set context or something [08:47:34] who knows. [08:47:43] yeah, didn't know what that meant [08:47:47] did you like my reinvention of html ? [08:48:22] axisys (~axisys@ip68-98-189-233.dc.dc.cox.net) joined #rest. [08:49:06] ?? [08:49:23] [08:49:36] vs [08:49:39] [08:49:49] ohhhh "on line to rule them all" [08:50:00] ohhhh "on _link_ to rule them all" [08:50:04] sh*t [08:50:07] :) [08:50:07] whatever [08:50:25] yeah, i actually started a mental exercise of designing a media type with a single affordance [08:50:33] xlink w/ inputs [08:50:38] theoretically you could serve a doc like that and XSLT it into html on the browser side right ? [08:50:56] just ot prove the point that while you _could_ do it all w/ a single element, it's ugly and not a good idea [08:51:19] mikekelly: absolutely, that was zackly what i mapped out as my "single-affordance" design! [08:51:20] well hal started like that right [08:51:28] really? [08:51:42] yeah if link was self-closing it was outbound [08:51:53] well, i was going to do the whole HTML.FORM thing, too. [08:51:57] if it contained other stuff it was equivalent of what is now [08:52:02] you never planned on getting that crazy, right? [08:52:08] yeah, i see your point [08:52:15] kevwilde_ (~kevwilde@igwe16.vub.ac.be) left irc: Quit: leaving [08:53:04] mamund: I have some plans for form'y stuff with HAL [08:53:13] cos I have another super cunning plan [08:53:17] {0.0} [08:53:29] {+.+} [08:53:32] I actually mentioned it on here the other day [08:53:38] talking to technopenis [08:53:48] Action: mamund checks to see if hell has frozen over [08:54:08] this is totally differnet though [08:54:14] this is submitting forms TO THE SERVER [08:54:21] DUN DUN DUUUNNNNN [08:54:32] you know web hooks ? [08:54:56] yep [08:55:43] well you can't really use them as a 3rd party of 2 services to glue them together [08:55:55] cos you just submit a URL callback that spews out a certain representation [08:56:09] ok [08:56:12] but if you also submitted a 'form' that built the query [08:56:26] hmmm [08:56:30] you coul actually use webhooks to properly plumb two services that didn't even know anything about each other [08:56:33] payloads for webhooks, then? [08:56:41] right [08:56:51] sounds similar to web intents [08:57:15] web intents are a client thing right ? [08:57:23] yes [08:59:02] KevBurnsJr (~KevBurnsJ@50.0.103.39) joined #rest. [09:03:16] gchristensen (~gchristen@unaffiliated/grahamc) left irc: Quit: Leaving... [09:12:51] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) left irc: Quit: Leaving. [09:39:05] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [09:48:01] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) joined #rest. [09:53:26] o/ [09:53:56] \o [09:54:20] mephju_ (~mephju@dslb-188-103-207-215.pools.arcor-ip.net) joined #rest. [09:54:20] mephju_ (~mephju@dslb-188-103-207-215.pools.arcor-ip.net) left irc: Client Quit [09:55:27] Noooooooooooooooooooooooooooooooooooooooooooo!!! Form free Hal forever. [09:56:32] up high mamund o/\o [09:57:26] O\/O down low whartung! [09:58:35] darrelmiller: Form ApplicationInterpretation Language [09:58:55] whartung: Exactly. [10:00:47] so the more i'm thinking about my book, the less I like it as a book. [10:01:03] i might do something smaller and more serial. [10:01:04] :) [10:01:09] to be honest. [10:01:20] steveklabnik: is this self-pub? [10:01:25] currently. [10:01:28] i like to maintain control. [10:01:37] at one time i was consdering a self-pub... [10:01:51] darrelmiller: it'll be another media type [10:01:52] i have friends whove been _very_ successful with it. [10:01:57] ooh.. a serial. With Heroes, and Big boobed damsels, and cliffhangers?? [10:02:05] lol [10:02:08] wanted to do it as a serial and allow folks to purchase units and/or whole thing [10:02:11] big boobs are an essential part of rest. [10:02:21] ... in the fanfiction anyway [10:02:37] mamund: was this for the html5 thing? [10:02:47] or something else? [10:02:50] you know what has been absolute fail, and a ruse pulled over the eyes of the public? [10:02:57] an early incarnation of the whol hypermedia stuff. [10:03:03] The concept of the "e-book -- buy now and get updates" [10:03:07] i had a hard time finding a publisher who wanted to do that project [10:03:10] never. happened. [10:03:12] ah. [10:03:24] whartung: really? pragprog does that. [10:03:28] whartung: subscribing to a book can make sense [10:03:35] not after the book hits print [10:03:46] no print, just e-book [10:03:59] mamund: ahh. it appears we really do think alike. Since i just picked up 'hypermediaapidesign.com' [10:04:04] whartung: yes, they do. [10:04:09] I've seen where you can get the "eearly release", like works in progress. [10:04:11] it's not SIGNIFICANT changes, i guess. [10:04:19] but you do get like second and third editions [10:04:20] darrel owns another good hypermedia domain [10:04:27] i know. [10:04:29] hypermedia-api? [10:04:31] what a catch. [10:04:37] no hyphen. [10:04:45] i own hypermediatypes [10:04:51] the double a is a bit annoying. [10:04:53] LOL [10:04:57] it really is [10:05:07] yeah [10:05:44] anyway, what i'm thinking of is doing a high-level treatse. like five or ten pages of the highest level topics, an introduction. and then giving that away, and pitching 'i dive into the details as a serial, here, and you can buy it.' [10:06:00] hmm.... [10:06:01] interesting [10:06:14] because ultimately i want everyone to know about this stuff. [10:06:18] yeah [10:06:22] so the high-level thing for free makes sense. [10:06:26] it feels right. [10:06:55] I'll just wait till it's avilable on torrent [10:07:03] do it. [10:07:03] :) [10:07:06] at one point was trying to think of a way to apply to books the "open-source" pattern of... [10:07:11] "code is for free" [10:07:20] if you want "support/consulting" call me [10:07:23] exactly. [10:07:29] so if i let people drive the topics I dive into... [10:07:32] it's almost like that. [10:07:36] hmmm [10:07:37] yeah [10:08:04] anyway, yeah. just thinking about this out loud. [10:08:17] distributing a book about hypermedia seems fundamentally worng. [10:08:18] wrong. [10:08:21] :) [10:08:24] LOL [10:08:30] its been bothering me. [10:09:31] I've seen early books promising much more than basic edits and typos fixes. Basically it seems that the author is so burnt from getting the initial book out that they give up when they find out they actually have to support and maintain the thing. [10:09:59] totally. [10:10:08] it's one of the reasons that i want to do it in smaller chunks [10:10:11] i thrive on feedback [10:10:18] i think a book might have too long of a feedback cycle for me. [10:20:38] nuclearsandwich (~nuclearsa@99.13.242.166) joined #rest. [10:34:53] kennethreitz (~kennethre@204.14.152.118) joined #rest. [10:37:50] steveklabnik: i used the blog for about a year or so to test thingsout, get feedback, etc. [10:38:15] Action: steveklabnik nods [10:39:35] carloseberhardt (~carlos@72.14.191.153) joined #rest. [10:39:49] kennethreitz (~kennethre@204.14.152.118) left irc: Quit: Textual IRC Client: http://www.textualapp.com/ [10:40:50] kennethreitz (~kennethre@204.14.152.118) joined #rest. [10:42:48] nuclearsandwich (~nuclearsa@99.13.242.166) left irc: Remote host closed the connection [10:46:04] mephju (~mephju@dslb-188-103-207-215.pools.arcor-ip.net) left irc: Read error: Connection reset by peer [10:55:08] nuclearsandwich (~nuclearsa@173.247.193.198) joined #rest. [10:59:01] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) left irc: Quit: Leaving. [11:19:43] bigbluehat (~bigblueha@host69-53-124-179.birch.net) joined #rest. [11:49:10] Action: mamund is checking out early, you're own your own.... [11:49:27] bye! [12:07:58] sbanwart (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) joined #rest. [12:08:12] bigbluehat (~bigblueha@host69-53-124-179.birch.net) left irc: Quit: Leaving. [12:34:46] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) joined #rest. [12:39:05] kennethreitz (~kennethre@204.14.152.118) left irc: Ping timeout: 252 seconds [13:10:13] \exit bye [13:10:16] \quit [13:10:18] carloseberhardt (~carlos@72.14.191.153) left irc: Quit: leaving [13:37:21] Nick change: scott- -> _scott [13:42:47] sbanwart_ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) joined #rest. [13:44:29] sbanwart (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) left irc: Ping timeout: 252 seconds [13:50:12] sbanwart__ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) joined #rest. [13:50:46] sbanwart_ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) left irc: Ping timeout: 268 seconds [13:59:59] bigbluehat (~bigblueha@adsl-74-177-96-212.gsp.bellsouth.net) left irc: Quit: Leaving. [14:07:25] kennethreitz (~kennethre@204.14.152.118) joined #rest. [14:15:46] sbanwart__ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) left irc: Remote host closed the connection [14:21:44] sbanwart__ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) joined #rest. [14:32:22] sbanwart__ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) left irc: Ping timeout: 248 seconds [14:32:34] sbanwart__ (~sbanwart@99-177-126-136.lightspeed.bcvloh.sbcglobal.net) joined #rest. [14:38:36] bradley-holt (~bradley-h@65-183-135-35-dhcp.burlingtontelecom.net) left irc: Quit: bradley-holt [14:52:33] grove (~grove@193.201.9.46.customer.cdi.no) left irc: Quit: grove [15:02:33] has anyone ever proposed a javascript API for setting/clearing the Authorization header ? [15:03:09] I mean a cross browser api like xhr is [15:07:46] gchristensen (~gchristen@unaffiliated/grahamc) left irc: Quit: Leaving... [15:09:38] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [15:15:00] so that javascript could "log out" someone? [15:18:50] right [15:19:27] plus an API for managing the cache [15:19:42] yea wouldn't that be nice [15:19:53] i.e. request disk allocation and purges [15:19:59] that kind of stuff [15:20:16] but, hey I can run a JVM in JS now...so load that, then run Java, and have it manage the conection for you! Yea, that's it! [15:20:27] Progress! [15:20:30] ;) [15:28:54] gchristensen (~gchristen@unaffiliated/grahamc) left irc: Quit: Leaving... [15:29:14] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [15:29:20] gchristensen (~gchristen@unaffiliated/grahamc) left irc: Client Quit [15:29:31] waaaay better than an applet [15:30:32] JS is awesome. Java is pretty awesome. Java in JS is uber-awesome. [15:30:45] like captain planet [15:31:03] he's the hero [15:31:12] Hey, you can run Rhino in the JVM...Rhino->Java->JVM->Rhino...SINGULARITY! [15:31:15] gunna take polution down to zero [15:31:24] oh wait.. jvm on js isnt' going to do that at all [15:31:40] also [15:31:49] wtf are jvm's ? [15:32:25] does it have anything to do with node.js? [15:32:28] You're a .NET guy, you wouldn't understand. [15:32:36] or Backbone.js [15:32:51] oooh...what's Backbone? I'LL BET IT'S AWESOME! [15:32:51] double tripple extra points if it's both [15:33:02] backbone is like [15:33:05] mvc [15:33:09] but like [15:33:14] on teh client [15:33:28] async mvc with js-sauce? [15:33:39] yes [15:33:52] awesome! [15:33:55] JVM = Javascript-View-Model iirc [15:34:01] haha [15:34:01] heh [15:34:19] i only care about clojure, really [15:34:25] and I haven't even touched that. [15:34:46] yet. [15:34:47] :) [15:35:12] That's good, Scala has been getting the evil eye recently. You don't want to be on that band wagon [15:35:24] orly? [15:35:31] what's the word round the water cooler? [15:36:09] i'm guessing it's not worse than resier-fs [15:36:11] is it ? [15:36:12] oh, folks are whining about it being difficult to use, citing a company that was all Scala rah-rah-sisk-boom-ba and are now "Java, you know, it works pretty good kthx" [15:36:17] no, not that bad lol [15:36:21] :D:D [15:36:43] the line was "Is Scala the new EJB 2" (which is kind of damning in java circles). [15:36:58] entity java beans [15:37:01] I know that one [15:37:06] "You know, coding isn't difficult enough --hey lets use Scala!" [15:37:35] lol [15:37:53] erlang. [15:38:02] yea [15:38:06] we're starting to use riak in a couple of places [15:38:38] looks like at some point someone is going to pull the short straw and have to figure out how to write a module for it [15:38:39] I mean, I don't know. I've not used it. I'm not a FP guy. I like Lisp and such, but I'm too ADD to want to be shackled by FP purity [15:38:48] riak looks cool. [15:39:00] steveklabnik: it's very cool but it's really just a key value store [15:39:01] mikekelly: yeah, there was the coda scandal with scala [15:39:04] in particular [15:39:06] isn't that a distribute db (yet another...) [15:39:17] whartung: yes, it's a Dynamo [15:39:20] yea [15:39:38] it's link walking stuff is map-reduce under the hood and apparently gets sluggish as you scale out [15:39:46] and the search is well.. [15:39:48] it's shit [15:39:50] Action: steveklabnik shruggs [15:39:56] i'm a Mongo kinda guy myself [15:39:57] Action: steveklabnik hides [15:40:13] I really like mongo for prototyping stuff [15:40:13] wow, crummy search on a key value store? Who'd have imagined [15:40:18] hahahah [15:40:25] what is incredible is elasticsearch [15:40:34] that thing is fucking magic. [15:40:37] know what that is? [15:40:40] it's Java [15:40:41] wtf. [15:41:01] havent heard of that, elasticsearch. what is/does it do? [15:41:20] it's a json doc database with search built in [15:41:24] that shards [15:41:33] indexes pretty much everything you throw in [15:41:36] oh [15:41:44] how is it different from Couch? [15:42:00] couch is map-reduce, it's not good for ad-hoc queries [15:42:05] like searches [15:42:11] sure [15:42:19] (well there's a lucene module, but we won't go there...) [15:42:37] some people have post-commit hooks that dupe data into elasticsearch from couch [15:42:45] ok [15:42:54] personally I'd opt for riak if I was gonna do that [15:43:13] I like couch conceptually. What I don't like is that it's CPU bound. A DB that's CPU bound is broken, IMHO [15:43:53] I haven't actually touched couchdb for a while now [15:43:53] there's nothing wrong with taking a map-reduce approach to run on multiple shards in parallel, but you certainly don't want to use map-reduce to run a filter function over every value in your store in a big sequential scan [15:44:12] smart index design still rules [15:44:27] yea. but that's what I like about couch is that it's all about index design [15:44:34] (which is why ad hoc sucks) [15:45:06] Action: fu-manchu is building an in-memory, column-oriented, index-everything database as we speak [15:45:25] in-memory elasticsearch ? [15:45:41] how does that work fu-manchu [15:45:50] REST [15:45:50] magic [15:45:52] JSON [15:45:55] JS [15:45:56] SCALABILITY [15:46:03] CAP THEOREM [15:46:18] I haven't looked in to column oriented. What's that bring to the table? [15:47:01] if most of your queries only touch a small subset of your columns, it results in much less data having to be loaded from disk and/or iterated over (and ignored) [15:47:18] how is that an advantage in a memory based db? [15:47:53] CLOUD [15:47:58] I forgot cloud [15:48:02] sorry [15:48:06] for this db, it's mostly an advantage in selecting matching rows [15:48:20] we generally select perhaps 3 columns out of 2000 [15:48:44] ah, you have a crap load of columns [15:49:01] yeah, big analytics [15:49:25] why not normalize those 3 columns out? or just separate them? [15:49:28] many of them rather sparse [15:49:35] no dbout lol [15:49:39] because it's a different 3 columns every time [15:49:48] ah [15:49:53] sinister [15:49:59] quite [15:50:09] fun though [15:50:16] makin good progress [15:50:59] so memory based for just hte 3 columns? [15:51:32] nope, all of it :) [15:51:47] how big is your heap? [15:52:06] that's a bit personal [15:52:09] haha [15:52:56] it's multiprocess [15:53:12] so are you using shared memory or something? [15:53:13] still in dev so we haven't tuned the sizes in production yet [15:53:24] what kind of numbers are you looking at? [15:53:25] no, sharding it all [15:54:09] about 5 billion data points right now [15:54:17] is it written in erlang ? [15:54:26] python! [15:54:30] because that's a pretty big feature in db's afaict [15:54:42] 2000 pts per row? [15:55:16] ints, strings, dna strands? [15:55:23] essentially...but for this first app, clever partitioning will bring that down a bit [15:55:33] eliminate some sparse regions [15:55:54] custom data structures or native python? [15:56:16] in-memory: native. disk backup: marshalled [15:56:49] just lists of packed bytes? [15:56:56] re-reading http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven is so interesting, now that i've spent a few months focused on REST. [15:57:02] like, the first time i read that, I was like 'wtf' [15:57:05] now i'm nodding my head. [15:57:08] feelsgoodman.jpg [15:57:13] its WTF to most people steveklabnik [15:57:29] whartung: effectively. we're using the standard library's "marshal" module, which packs in its own way [15:57:56] I only ask [15:58:01] because we did a bunch of caching here [15:58:10] in java [15:58:39] and a Map for 25M entries was, like, 2G of RAM. Somethng truly crazy makin's. [15:58:41] steveklabnik: yeah Roy's writing is pretty dense [15:58:44] or nuanced [15:58:53] or some other adjective that makes me sound like I know what I'm talking about [15:59:00] gchristensen (~gchristen@unaffiliated/grahamc) joined #rest. [15:59:35] mamund: Ian got back to me on that discussion on Stefan's blog [16:00:05] maxist ;) [16:00:10] marxist [16:00:16] our ops guys would like us to try FusionIO to cut down the RAM, but...we've been having trouble getting a trial version ;) [16:00:16] pinko [16:00:20] :) [16:00:43] I've not heard of FusionIO [16:01:15]