Collaboration with Autistici-Inventati Resistant Network

Attached to this page is the A/I "Orange Book" which describes how they have set up their resistant network. I chatted with a couple of them and I think we could benefit from their CFengine ideas (and others) and the could benefit from our use of ICE. An English version of the Orange Book is attached and a transcript of our chat together follows.

09:14 <&bomboclat> porcodio
09:15 <&lesion> eh quante ne vuoi
09:15 < Avvelenato> perch`e usare skype???
09:15 <&bomboclat> nel 2008 voglio delle interfaccie grafiche
09:15 < Avvelenato> non puoi usare quei grazioni programmi opensource?
09:15 <&bomboclat> Avvelenato, per provare la webcam?
09:15 <&bomboclat> FUORI I NOMI
09:15 <&bomboclat> a parte ekiga
09:15 < Avvelenato> :p
09:15 <&bomboclat> io voglio streammare su icecast 2
09:15 <&bomboclat> la mia faccia di cazzo
09:15 < Avvelenato> io non uso skipe.. ma sono molti i programmi che lo emulano
09:15 <&bomboclat> che programma uso GPL ?
09:15 < Avvelenato> si
09:30 -!- bomboclat [bomboclat@mufhd0-95E6567E.fastres.net] has quit [Leaving]
09:35 < Avvelenato> Porco dio cane onnipotente
09:36 < Avvelenato> SApete per caso se sono io ad avere problemi con il .zip  o `e noblogs?
09:54 -!- babau [rizomo@mufhd0-EB72053.t2.dsl.vodafone.it] has quit [Input/output error]
10:02 -!- Avvelenato [dario@mufhd0-A6E2E5B1.cust-adsl.tiscali.it] has quit [Ping timeout]
10:13 < ryan> hello hello
10:13 <&lesion> hello
10:15 < ryan> buscando mais info di Plan R*
10:15 <&lesion> que info ?
10:16 < ryan> specifiches
10:17 < ryan> stiamo lavorando con un progetto simile
10:17 < ryan> -> http://dev.bunke.indymedia.org/
10:18 < ryan> http://linefeed.org/
10:19 <&lesion> :D
10:20 <&lesion> so what kind of info do you need ?
10:20 < ryan> posso falar em portugues melhor que italiano :)
10:20 <&lesion> eh porugues is a problem. what about english ?
10:20 < ryan> yes, english is fine
10:20 <&lesion> nice
10:20 < ryan> i'm interested in architecture details
10:21 <&lesion> have you seen the pdf ?
10:21 < ryan> no!
10:21 <&lesion> the orange book I mean
10:21 < ryan> i dont think so!
10:24 <&lesion> ok wait
10:25 < nah> http://dev.autistici.org/orangebook/ :x
10:25 <&lesion> http://dev.autistici.org/orangebook/index.html.en
10:25 <&lesion> yep
10:25 < nah> :)
10:25 <&lesion> thanks :P
10:26 < nah> disponha
10:27 < ryan> mm thanks
10:28 < ryan> this is interesting..
10:28 < ryan> our plan involves the use of ICE
10:28 < ryan> http://www.zeroc.com/ice.html
10:32 < ryan> i think 8.1.3 is mostly where we are focused
10:33 < ryan> because we want to build a network where all IMC sites are hosted on all the servers in the network
10:33 < ryan> interesting
10:34 < ryan> not exactly what we have planned but -we- can definitely use parts of what you have (especially the cfengine stuff)
10:34 < ryan> and this orange book could contain some of the parts that we have (especially the ICE stuff)
10:34 < ryan> is this being worked on?
10:35 < ryan> is this network implemented in any way?
10:35 <&megabug> autistici/inventati is a working example
10:36 <&megabug> lesion ! tesoro :)
10:36 < ryan> how many servers are in the network?
10:36 <&megabug> 9 actually
10:37 <&megabug> 4 are the main server for services like web mail etc.
10:37 <&megabug> and 5 servers are for backup / monitor / statistics and minor services
10:37 <&lesion> 5
10:37 <&megabug> ah! right 5 :P
10:38 < ryan> are any of them supporting italy indymedia?
10:38 <&lesion> nope
10:38 <&lesion> to host a web service replicated on a lot of hosts the key point is replicate database stuff
10:38 < ryan> our main objective is to build a network such that if we suffer server seizure (by law enforcement), hardware failure or donation loss (like ahimsa),
10:39 < ryan> there is no interruption in service
10:39 <&lesion> cool
10:39 < ryan> yes, we manage the data with ICE
10:39 < ryan> so ICE is an object-store for us
10:39 < ryan> using a database as a backup
10:40 <&lesion> but have you developed the indymedia cms to use ice ?
10:40 < ryan> well, this is what we're working on
10:40 < ryan> we decided to merge the indymedia CMS teams
10:40 <&lesion> cool
10:40 < ryan> at least, mir & sf-active
10:40 < ryan> because we were stretched too thin
10:40 <&lesion> nice idea :D
10:41 <&megabug> argh :)
10:41 < ryan> so we are working on a new front end, which i believe will be something like joomla, extended by cakephp
10:41 <&lesion> noooooooooo joomla buuuuuuuuu
10:41 <&lesion> :)
10:41 < ryan> why not?
10:42 < ryan> the days of mir and sf-active are numbered
10:42 < ryan> soon, they will be only memories
10:42 <&megabug> how much time take a new front end to be developed?
10:43 < ryan> well, if we begin with an existing one, not much time at all :)
10:43 < ryan> our point was to find a CMS which already has a large development community
10:43 < ryan> but, none of them were very good
10:43 < ryan> so, then we thought we would write our own, using CakePHP
10:44 < ryan> but, we didnt want to write from scratch
10:44 < ryan> then, we found a Joomla<->CakePHP bridge 
10:44 < ryan> and i think we will use that 
10:44 < ryan> so we can get all the benefits of the large development community of joomla AND cakephp
10:45 <&lesion> adapting joomla to your needs will need more time than develop a new front end from scratch imho
10:45 < ryan> how so?
10:45 <&lesion> eh you will see ;)
10:45 < ryan> nah
10:46 <&lesion> hope so
10:46 <&megabug> uhm i don't now, how much joomla is good for you?
10:46 < ryan> who will make the admin interface for our cms-from-scratch?
10:46 < ryan> design the nice interface?
10:46 < ryan> make all the extensions?
10:46 < ryan> etc etc
10:46 < ryan> i've written an indymedia cms from scratch before, so i know how it goes
10:46 < ryan> for instance -> http://extensions.joomla.org/
10:46 <&lesion> ok but now you have experience with you :D
10:46 <&megabug> maybe you can write plugins
10:47 < ryan> that would -never- happen with our own cms
10:47 < ryan> yes, and my experience tells me: reusable code is king :)
10:47 <&megabug> to giva joomla the features needed by imc
10:48 <&megabug> give
10:48 <&lesion> but also, modify others code is a pain 
10:48 < ryan> but, what features an imc needs that joomla does not have?
10:49 <&megabug> i don't know :)
10:49 < ryan> none!
10:49 < ryan> in fact, it offers much more than we could build ourselves
10:49 <&megabug> you tell us that you want to developo with cakephp :)
10:49 <&megabug> and joomla
10:49 < ryan> we want to extend with cakephp
10:50 < ryan> to develop the ICE interface
10:50 <&lesion> ah diobono vero
10:50 < ryan> and, if we want to add new features, we can do it
10:50 <&megabug> ah ok
10:50 < ryan> for instance, joomla currently has 27 extensions for listing articles
10:50 <&megabug> but why ice?
10:51 <&lesion> to spread contents over servers
10:51 < ryan> i havent found something else as good as ice
10:51 <&lesion> mysql replication ? 
10:52 < ryan> we will also use mysql replication
10:52 < ryan> but ice will act between the web server tier and the db tier
10:52 < ryan> for instance, mysql is a poor and slow object store
10:52 < ryan> ice is a fast object store
10:54 <&megabug> you want to switch from mysql to ice?
10:54 <&megabug> i don't know how ice does really
10:54 < ryan> ice is middleware
10:55 < ryan> it sits between the web tier and the db tier
10:55 < ryan> so, for instance, if a user logs into Web Server #10
10:55 < ryan> it stores the session, via ice, in mysql but ice remembers it
10:55 < ryan> so, when the user returns to Web Server #4, and asks for the session,
10:55 <&megabug> ok
10:55 < ryan> ice gives it the session object
10:56 < ryan> without going to mysql
10:56 <&megabug> ah interesting
10:56 < ryan> ice also understands where all the webservers are
10:56 <&megabug> but in your replication model
10:56 < ryan> and where all the dbservers are
10:57 <&megabug> replicating sessions is a different level from replicating data
10:57 < ryan> what do you mean?
10:57 <&megabug> sessions are not stored on mysql
10:58 < ryan> we intend to
10:58 <&megabug> is a web server issue
10:58 <&megabug> we got the same session problem with our services
10:59 < ryan> i read about it in the orange book
10:59 < ryan> you use sticky sessions
10:59 < ryan> but if you store the sessions in the db, you dont have this problem
10:59 <&megabug> yes maybe we can reconsider the solution
10:59 < ryan> yes, sticky sessions mess up your load balancing
11:00 <&megabug> mmmhh, not really
11:00 <&megabug> with an high number of users the difference is not significant
11:00 < ryan> well, the point of round-robin dns is so that hits to the server can be all over the place, right?
11:01 <&megabug> yep
11:01 < ryan> but if you have a number of heavy users "stuck" to a particular web server, then you have defeated this purpose
11:02 < ryan> by "heavy users" i mean... 4-5 users uploading 50MB videos to an imc all at once
11:02 <&megabug> yep but its the worst case scenario
11:02 <&megabug> its very unlike to happen
11:02 < ryan> i think it would be quite common with IMC, especially during a period of heavy usage (like a big protest)
11:02 < ryan> how unlikely it is to happen depends on how many webservers you have :)
11:02 <&megabug> and users...
11:02 < ryan> if it is 2, it is very likely to happen
11:03 <&megabug> uhm
11:03 <&megabug> it's 1 possibility on 2^users
11:03 < ryan> well.. our goal is to host over 140 websites on the network
11:03 <&megabug> (when all the 5 uploaders match the same server)
11:03 < ryan> all of them receiving or sending large media files
11:04 < ryan> so we want to make sure that we get the advantage of round-robin dns
11:04 <&megabug> the balancing with rrdns is very good
11:04 <&megabug> the problem is another
11:05 <&megabug> and is that not all the user make the same use of services
11:05 <&megabug> so you got 500 lurker and 1 heavy uploader for example
11:06 <&megabug> in this case this single uploader can exausth all resources on a server
11:07 <&megabug> but it's very rare, we got similar problem 6/7 times in 5 years
11:08 < ryan> sure, that could happen
11:08 < ryan> but it's even more likely to happen with the model we have now which is 15 IMC's on one webserver :)
11:08 <&megabug> ghghghgh
11:08 <&megabug> yes i imagine :D
11:09 < ryan> well, this is interesting.. i think we will definitely want to use your cfengine stuff
11:10 < ryan> and maybe you will be interested in how we use ICE for web applications
11:10 <&lesion> yep
11:10 <&lesion> we have to replicate lifetype
11:10 <&megabug> yep, for our front-end maybe interesting
11:10 < ryan> who are the main people who work on this for a/i?
11:10 <&megabug> for user websites maybe not :) but we'll see
11:10 <&lesion> on cfengine stuff? 
11:11 < ryan> well, yeah, just the whole idea of the resistant network?
11:11 -!- lesion [lesion@mufhd0-6BDA3380.ip.fastwebnet.it] has quit [.]
11:11 <&megabug> the r* come from A/I collective altogheter
11:11 < ryan> also do you mind if i share this chat with other people in our group working on this?
11:12 <&megabug> from discussion internal to the group
11:12 <&megabug> yes of course
11:12 <&megabug> all R* stuff is public :)
11:12 < ryan> nice, same with our stuff
11:12 < ryan> also we are meeting in rio de janeiro 
11:12 < ryan> in september
11:13 < ryan> to advance this project
11:13 < ryan> at techmeet
11:13 < ryan> (techmeet.org)
11:13 < ryan> the groups involved in this project are:
11:13 <&megabug> cool :)
11:13 < ryan> techmeet.org
11:13 < ryan> linefeed.org
11:13 < ryan> imc-cms group in indymedia, lists.indymedia.org/imc-cms
11:13 < ryan> dev.bunke.indymedia.org
11:14 < ryan> come to rio!
11:14 < ryan> :)