#archiveteam 2014-11-10,Mon

↑back Search

Time Nickname Message
00:26 🔗 Ymgve has quit IRC ()
00:33 🔗 Start is now known as StartAway
00:33 🔗 schbirid2 has joined #archiveteam
00:35 🔗 schbirid has quit IRC (Read error: Operation timed out)
00:44 🔗 wp494 has joined #archiveteam
00:44 🔗 BiggieJo1 has joined #archiveteam
00:47 🔗 BiggieJon has quit IRC (Read error: Operation timed out)
00:53 🔗 Mayonaise has quit IRC (Ping timeout: 365 seconds)
00:53 🔗 zenguy_pc has quit IRC (Read error: Operation timed out)
01:06 🔗 primus104 has quit IRC (Leaving.)
01:09 🔗 zenguy_pc has joined #archiveteam
01:38 🔗 Mayonaise has joined #archiveteam
01:40 🔗 StartAway is now known as Start
01:57 🔗 xk_id has quit IRC (Ping timeout: 620 seconds)
02:06 🔗 Mayonaise has quit IRC (Ping timeout: 365 seconds)
02:11 🔗 Mayonaise has joined #archiveteam
02:35 🔗 logchfoo starts logging #archiveteam at Mon Nov 10 02:35:30 2014
02:35 🔗 logchfoo has joined #archiveteam
02:40 🔗 ex-parrot has joined #archiveteam
02:52 🔗 Ravenloft has quit IRC (Ping timeout: 606 seconds)
02:57 🔗 bauruine has quit IRC (Ping timeout: 265 seconds)
03:03 🔗 schbirid2 has quit IRC (Read error: Operation timed out)
03:03 🔗 bauruine has joined #archiveteam
03:11 🔗 schbirid2 has joined #archiveteam
03:47 🔗 mistym has joined #archiveteam
03:51 🔗 Lord_Nigh has quit IRC (Read error: Operation timed out)
03:53 🔗 Lord_Nigh has joined #archiveteam
04:23 🔗 bsmith093 has quit IRC (Read error: Operation timed out)
04:38 🔗 mistym has quit IRC (Remote host closed the connection)
04:39 🔗 bsmith093 has joined #archiveteam
04:53 🔗 ex-parrot has quit IRC (Leaving.)
04:56 🔗 aaaaaaaaa has quit IRC (Leaving)
05:23 🔗 mistym has joined #archiveteam
05:33 🔗 JonimusP is now known as Jonimus
06:08 🔗 Start is now known as StartAway
06:08 🔗 signius has quit IRC (Ping timeout: 480 seconds)
06:18 🔗 signius has joined #archiveteam
07:09 🔗 ruukasu has quit IRC (Quit: WeeChat 1.0.1)
07:14 🔗 ruukasu has joined #archiveteam
07:20 🔗 SketchCow arkiver: chfoo: Is SWIPNET data done being uploaded, or is that still happening?
07:23 🔗 primus104 has joined #archiveteam
07:39 🔗 rduser has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 SadDM has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 twrist has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 matthusb_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 altlabel has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 pikhq has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ionpulse has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Jogie has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 everdred has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 eprillios has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Insomnia_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 gibigiana has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Aranje has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 dcmorton has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 SmileyG has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Cameron_D has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 slash` has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 pft has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Sanqui has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 antomatic has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 khaoohs has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Sue_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 mistym has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Lord_Nigh has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 GLaDOS has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 arkiver has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Shank_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 RainbowCo has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 fluff has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 lhobas has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 blergh has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Nemo_bis has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Riviera has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 SN4T14__ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ats has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 aNthraXx has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 brayden has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Zebranky_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Deewiant_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Atluxity has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 tfgbd has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 namespace has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ruukasu has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 bauruine has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 pfallenop has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Sellyme_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 dashcloud has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Elegance has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 PepsiMax has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 tephra_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 DFJustin has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Sk1d has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 NovaKing_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 zifnab06 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 danneh_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 soultcer has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Kniffy has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 primus104 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 schbirid2 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Coderjoe has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 norbert79 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 is- has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 nox has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ersi has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ZorbaBeta has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 garyrh has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 will__ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Void_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Boppen has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Kenshin has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 wp494 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 trs80 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 filippo_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 mhazinsk has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 bsmith093 has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 SketchCow has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 kanzure has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 lytv has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 RedType has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 ryan___ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 chfoo has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 dx has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 xmc has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 zenguy_pc has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 BlueMaxim has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 StartAway has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 primus_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Jonimus has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Meeh_ has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 swebb has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 lemonkey has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 w0rp has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 RealMarc has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 lukeman has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 dserodio has quit IRC (ircd.choopa.net hub.efnet.us)
07:39 🔗 Laverne has quit IRC (ircd.choopa.net hub.efnet.us)
17:03 🔗 logchfoo starts logging #archiveteam at Mon Nov 10 17:03:04 2014
17:03 🔗 logchfoo has joined #archiveteam
17:03 🔗 vman315 has joined #archiveteam
17:04 🔗 vman315 has quit IRC (Client Quit)
17:05 🔗 nertzy has quit IRC (This computer has gone to sleep)
17:06 🔗 average has joined #archiveteam
17:06 🔗 average hello archiveteam
17:07 🔗 average i've been watching your activities
17:07 🔗 average i am proud of you
17:07 🔗 average you're amazing
17:07 🔗 * schbirid2 fetches the glitter box
17:08 🔗 average may I ask a question related to the Twitter stream dump ?
17:08 🔗 average for example this one https://archive.org/details/archiveteam-twitter-stream-2014-09
17:08 🔗 average (btw I'm from Romania, so I'm not an english speaker, please excuse my grammar or spelling errors)
17:10 🔗 average the Twitter stream dump. uhm, how is it produced ?
17:10 🔗 average I mean, how do you consume it ? I've been able to use it as well
17:10 🔗 chfoo arkiver SketchCow: swipnet should be done. it hasn't been in the tracker for a while.
17:10 🔗 average with the api endpoint https://stream.twitter.com/1.1/statuses/filter.json
17:11 🔗 average and the parameters delimited=length&track=romania
17:11 🔗 average I've seen that archiveteam has a github account with all the code that goes into pulling this data
17:11 🔗 average I was unable to find the code for the Twitter stream dumper
17:11 🔗 average could anyone please point me to it ?
17:12 🔗 SmileyG i don't think we are able to share that
17:12 🔗 average can you talk about it then ?
17:12 🔗 average if sharing is not an option
17:13 🔗 average SmileyG: ^^
17:13 🔗 SmileyG hmmmm i think we can talk about it - however i personally don't know much about it
17:13 🔗 SmileyG other than we have some special access to something
17:14 🔗 average ah I see. so basically if I were to try to do what you're doing, I wouldn't be able to, right ?
17:14 🔗 chfoo espes__: if i recall correctly, the scripts paginated everything on the homepage including friends list. but that's all saved in the warcs and difficult to get back out.
17:14 🔗 mistym has quit IRC (Remote host closed the connection)
17:15 🔗 SmileyG average: yeah pretty much :(
17:16 🔗 average SmileyG: does this mean you have access to stream.twitter.com/1.1/statuses/firehose.json ?
17:16 🔗 average SmileyG: aka firehose (https://dev.twitter.com/streaming/firehose)
17:16 🔗 SmileyG I don't think we have full access no
17:16 🔗 average I see
17:16 🔗 SmileyG i believe we get it via somewhere else.
17:16 🔗 joepie91 .... okay, wow
17:16 🔗 SmileyG but they do, or something
17:16 🔗 joepie91 so I accidentally ended up on the Hyves Games forums
17:16 🔗 * SmileyG isn't sure. This is all speculation.
17:16 🔗 joepie91 "to keep the forums clean, this topic will be locked and eventually removed"
17:16 🔗 joepie91 DO THESE PEOPLE NEVER FUCKING LEARN
17:17 🔗 schbirid2 https://archive.org/details/twitterstream
17:17 🔗 average schbirid2: yes, I know about that
17:17 🔗 joepie91 is this the first time an archiveteam target is deleting data a *second* time?
17:17 🔗 schbirid2 average: it says spritzer right there
17:17 🔗 joepie91 cc midas
17:17 🔗 average schbirid2: ah ! where's spritzer ?
17:18 🔗 average SmileyG: you guys, like all the archiveteam, are a hidden jewel of the interwebz
17:18 🔗 average I swear
17:18 🔗 SmileyG right, gotta go.
17:19 🔗 average ok
17:25 🔗 Trskl has joined #archiveteam
17:26 🔗 espes__ is there a api/tool to pull a specific file from a warc?
17:27 🔗 joepie91 espes__: there's a bunch, I don't think any of them support doing it directly from IA though
17:28 🔗 everdred Hi all. I didn't see a wiki page and was wondering if anyone had considered archiving Yahoo Directory before it shuts down in December.
17:29 🔗 everdred (If you'll recall, before Yahoo did the search engine thing, the manually-curated directory *was* the original Yahoo product.)
17:29 🔗 everdred It's set to disappear on 12/31. http://yahoo.tumblr.com/post/98474044364/progress-report-continued-product-focus
17:31 🔗 logchfoo starts logging #archiveteam at Mon Nov 10 17:31:59 2014
17:31 🔗 logchfoo has joined #archiveteam
17:32 🔗 mistym has joined #archiveteam
17:49 🔗 SmileyG I think we've already crawled it....
17:51 🔗 aaaaaaaaa They had some server problems on the first few attempts. I don't know if they tried lately.
17:51 🔗 primus104 has quit IRC (Leaving.)
17:53 🔗 aaaaaaaaa Wow, that was unclear. Yahoo was having server problems the first few times it was tried. I don't know if those were fixed or if there have been any recent attempts to get it.
17:54 🔗 SmileyG D:
17:54 🔗 SmileyG so basically we broke yahoo?
17:56 🔗 bobby_ has joined #archiveteam
18:00 🔗 kahayes has quit IRC (Read error: Connection reset by peer)
18:04 🔗 everdred I see. Basically, just wanted to come by and shine some light on Directory in case its closing hadn't been noticed before. I don't really have the skill set to troubleshoot; I'm more of a random sympathetic party who tries to run a seesaw script now and then to help out. :)
18:05 🔗 everdred I'll trust that more capable minds can figure out whether it's doable.
18:10 🔗 SadDM So my Dark Horse forum grab finished, but it looks like it's probably incomplete.
18:10 🔗 SadDM http://boards.darkhorse.com/ indicates that there are 17491 topics, which means that there should be *at least* that many urls fetched...
18:10 🔗 SadDM but the phpbb3 lua script says that it grabbed less than 10350
18:15 🔗 joepie91 everdred: all help is very welcome :D
18:17 🔗 SketchCow Internet Archive inside is running to an Agile model.
18:17 🔗 SketchCow That should be.... interesting.
18:17 🔗 SadDM zcat boards.darkhorse.com-20141110.warc.gz | grep 'WARC-Type: request'|wc indicates that there are 10158 files in the warc :-(
18:17 🔗 SketchCow About to find out my role in a "Content Scrum"
18:20 🔗 tfgbd how do I rename a file if it isn't showing up in the file editor?
18:21 🔗 tfgbd I just uploaded extra stuff with the python uploader and they aren't showing up in the list :(
18:23 🔗 bobby__ has joined #archiveteam
18:25 🔗 joepie91 SketchCow: oh dear
18:25 🔗 joepie91 :)
18:25 🔗 joepie91 well to be fair, the ideas behind agile can work really well... just most organizations/people don't really seem to implement them correctly
18:26 🔗 bobby_ has quit IRC (Ping timeout: 240 seconds)
18:26 🔗 joepie91 let's hope that works better here
18:26 🔗 * joepie91 resumes writing hyves parsing code
18:38 🔗 DFJustin tfgbd: if you just uploaded it, you might have to wait for the jobs to finish running
18:56 🔗 xmc joepie91> is this the first time an archiveteam target is deleting data a *second* time? <-- i would find it hard to believe that it would have taken this long.
19:05 🔗 primus104 has joined #archiveteam
19:07 🔗 joepie91 xmc: same, but I can't think of any prior instance...
19:08 🔗 xmc i'm almost certain there was something about a year ago
19:08 🔗 xmc but i can't remember more detailedly than that
19:42 🔗 Aranje has quit IRC (Quit: Three sheets to the wind)
20:06 🔗 ex-parrot has joined #archiveteam
20:08 🔗 ex-parrot has quit IRC (Client Quit)
20:19 🔗 nertzy has joined #archiveteam
20:23 🔗 Panasonic has quit IRC (Ping timeout: 480 seconds)
20:24 🔗 bobby__ has quit IRC (Ping timeout: 240 seconds)
20:30 🔗 bobby_ has joined #archiveteam
21:01 🔗 nertzy has quit IRC (This computer has gone to sleep)
21:03 🔗 arkiver chfoo: you're keeping some great logs with logchfoo
21:03 🔗 arkiver Are you thinking about grabbing more channels?
21:09 🔗 bobby_ has quit IRC (Quit: Page closed)
21:10 🔗 Bobby_ has joined #archiveteam
21:28 🔗 BlueMaxim has joined #archiveteam
21:31 🔗 mistym has quit IRC (Remote host closed the connection)
21:44 🔗 chfoo arkiver: what other channels do you have in mind?
21:46 🔗 Bobby_ has quit IRC ()
21:52 🔗 mistym has joined #archiveteam
21:53 🔗 xmc what's the link to your archives, chfoo?
21:53 🔗 xmc i'm bad at website.
21:53 🔗 Start http://archive.fart.website/bin/irclogger_logs
21:54 🔗 xmc thx
21:55 🔗 xmc i see #quitpic , #oodbye missing
21:55 🔗 xmc i'm not in any other not-in-fartwebsite project channels atm
21:56 🔗 xmc but i think all the project channels should be logged
21:57 🔗 Start agreed
22:05 🔗 arkiver chfoo: I think it would be great if you are able to log all the channels
22:12 🔗 Nemo_bis joins may be considered unnecessarily noisy
22:22 🔗 mistym has quit IRC (Remote host closed the connection)
22:37 🔗 mistym has joined #archiveteam
22:54 🔗 RedType has quit IRC (Quit: leaving)
22:54 🔗 RedType has joined #archiveteam
22:54 🔗 RedType has quit IRC (Client Quit)
23:07 🔗 RedType has joined #archiveteam
23:12 🔗 Ymgve__ has joined #archiveteam
23:15 🔗 nertzy has joined #archiveteam
23:16 🔗 Ymgve has quit IRC (Ping timeout: 512 seconds)
23:36 🔗 nertzy has quit IRC (This computer has gone to sleep)

irclogger-viewer