| Time |
Nickname |
Message |
|
00:17
π
|
|
BlueMax has joined #archiveteam-bs |
|
00:31
π
|
|
Docti has joined #archiveteam-bs |
|
00:33
π
|
Docti |
Hello, here I am for further discusion |
|
00:33
π
|
JAA |
Docti: Is there a list of the forums that will be deleted? They only listed the ones that will live on, if I understand that correctly. |
|
00:35
π
|
Docti |
You are right, they only listed what will not be deleted |
|
00:36
π
|
Docti |
Here is the list of all the categories. If they are not in the list of what will be kept, they will be deleted https://forum.doctissimo.fr/doctissimo/liste_categorie.htm |
|
00:37
π
|
Docti |
But, since they might have changed the name of what will be kept (because they might rename it or put it in new categories), perhaps it would be easier to save everything ? |
|
00:37
π
|
JAA |
Wow, 320 million messages in total (across all forums, not just the sexuality ones). |
|
00:38
π
|
JAA |
I can try, but chances are their servers won't be able to handle that much in 4 days. |
|
00:40
π
|
Docti |
Thank you |
|
00:40
π
|
Docti |
So far I believe only the Sexuality forums are in danger, because they are not "socially acceptable" |
|
00:40
π
|
Docti |
I believe the other categories about family, daily life, medicine, etc, are safe |
|
00:40
π
|
JAA |
They're really going out of their way to make it hard to access the content. Some threads are not linked directly but their URLs are hidden in "cryptlinks" (which are just base64-encoded). |
|
00:41
π
|
JAA |
Yeah, I will target only the sexuality forums. |
|
00:42
π
|
Docti |
Perfect :) I have to admit I have very little knowledge in that domain of computer science. I am sure you will do your best |
|
00:43
π
|
JAA |
There are still 54 million posts just in these forums. This will be a challenge. |
|
00:43
π
|
JAA |
(For their servers) |
|
00:48
π
|
Docti |
This is much more than what I thought ! I hope the servers will be able to handle it |
|
01:05
π
|
JAA |
Docti: I can't find "Troubles de lβΓ©rection" and "Ejaculation prΓ©maturΓ©e" on https://forum.doctissimo.fr/doctissimo/liste_categorie.htm. They're listed as remaining in the announcement. |
|
01:12
π
|
Docti |
Indeed, I cannot find them either. Perhaps they have already been moved to their new categories ? |
|
01:13
π
|
Docti |
By the way, forget the part "Exhibition et voyeurs" or put it with the lowest priority - lots of post and topics of very low quality |
|
01:14
π
|
JAA |
Well, it seems that the ones that were already moved are still linked. E.g. "Contraception" is linked to the SantΓ© part of the forums, not in SexualitΓ© anymore. |
|
01:15
π
|
JAA |
Only some of them, apparently. http://forum.doctissimo.fr/doctissimo/erection/liste_sujet-1.htm exists and redirects to https://forum.doctissimo.fr/sante/Troubles-de-l-erection/liste_sujet-1.htm |
|
01:16
π
|
JAA |
Yeah, the other one was moved as well: https://forum.doctissimo.fr/sante/Ejaculation-prematuree-ou-precoce/liste_sujet-1.htm |
|
01:20
π
|
Docti |
Indeed, nice find, they have already been moved, but not the others. Just so you know: IST = Infections sexuellement transmissibles |
|
01:23
π
|
JAA |
Yes, some have been moved and are linking to the new location, some have been moved and are not linked anymore, and some have not been moved yet. |
|
01:24
π
|
JAA |
Yep, figured that one out, thanks! :-) |
|
01:26
π
|
|
mtntmnky has quit IRC (Remote host closed the connection) |
|
01:27
π
|
|
mtntmnky has joined #archiveteam-bs |
|
01:27
π
|
JAA |
There's also one which is in a different section but not listed in the announcement ("Techniques de sΓ©duction"). |
|
01:28
π
|
JAA |
It appears that it wasn't moved recently but was already in the psychology section before. |
|
01:29
π
|
JAA |
Some of the ones that will stay were also already elsewhere. |
|
01:31
π
|
Docti |
Great, they will not be deleted, so this is something you will not have to save |
|
01:33
π
|
JAA |
Oh fantastic, they didn't even set up redirects for the former locations of the moved categories. |
|
01:45
π
|
Docti |
That's nice :) If you don't mind, I will have to leave, it is a bit too late for me. I believe you can contact me on my page on the archiveteam site in the meantime. Thank a lot you for you work ! |
|
01:46
π
|
JAA |
Yeah, I should leave as well, but I'll see if I can get it running before. |
|
01:46
π
|
JAA |
Good night! |
|
01:47
π
|
Docti |
Good night |
|
01:53
π
|
|
Docti has quit IRC (Ping timeout: 252 seconds) |
|
01:59
π
|
|
Nikchemny has joined #archiveteam-bs |
|
02:00
π
|
Nikchemny |
JAA: Hello, are there news about archive.st? |
|
02:00
π
|
Nikchemny |
Btw, do AT use Alexa toolbar? |
|
02:01
π
|
JAA |
Nikchemny: No, I've been busy archiving things that disappear in the next few days. |
|
02:02
π
|
Nikchemny |
Which for example? |
|
02:04
π
|
JAA |
Bitcoin Forum, Kongregate forums, now Doctissimo forums. |
|
02:04
π
|
Nikchemny |
Hm, that's interesting. Are they on AB? |
|
02:05
π
|
JAA |
No |
|
02:05
π
|
JAA |
That's why I've been busy and still am. |
|
02:21
π
|
PovAddict |
bitcoin forum really? |
|
02:29
π
|
JAA |
Forum goes down, forum gets saved. |
|
02:32
π
|
Nikchemny |
I think everyone must download https://www.alexa.com/toolbar to thank Brewster Kahle. |
|
02:33
π
|
JAA |
No thanks, especially since it hasn't been owned by Brewster in ... 20 years? |
|
02:34
π
|
Nikchemny |
Yes, I know, but he created it... |
|
02:34
π
|
JAA |
So? I'd rather thank him for creating IA by donating there. |
|
02:35
π
|
JAA |
Than install adware/spyware/whatever the hell that toolbar probably is. |
|
02:36
π
|
Nikchemny |
Do you think that Amazon is watching for the Alexa toolbar users? |
|
02:36
π
|
Nikchemny |
Btw, I realized that fart.website is 17,000 in Russia https://www.alexa.com/siteinfo/fart.website |
|
02:36
π
|
JAA |
You bet they're using it to determine your interests and serve the corresponding Amazon ads. |
|
02:37
π
|
JAA |
In any case, this is very off-topic. -> #archiveteam-ot |
|
02:37
π
|
Nikchemny |
Ok |
|
02:37
π
|
JAA |
I think my Doctissimo script is ready, just running some tests. |
|
02:41
π
|
JAA |
Yep, seems fine. Let's set some servers on fire. |
|
02:46
π
|
JAA |
There are 13857 pages listing 50 topics each across all forums I intend to grab now (everything in the SexualitΓ© section). So that's 679k topics. Average post count per topic is about 80, so that's "only" around 1.4 million requests in total. Not as bad as expected. |
|
02:47
π
|
JAA |
Er, typo, 693k topics |
|
02:47
π
|
Nikchemny |
JAA: Btw, what's with Telegram? Is it saves good or some content is missing (Mhm, videos and audios can't be saved)? |
|
02:49
π
|
JAA |
Nikchemny: Never looked into it in detail. Can you check some of the accounts we've archived with ArchiveBot in the WBM? |
|
02:52
π
|
Nikchemny |
Eghrm. You saved it like t.me/something? Em, it can't be looked good. If the start link would be t.me/s/something then There would be the last 20 posts and I can tell which number of post is the last. Hm, JAA, I'll look for COVID-channels, I mean for their pics |
|
02:55
π
|
Nikchemny |
JAA: Does it save posts like t.me/s/something/number or t.me/something/number? |
|
02:56
π
|
JAA |
Nikchemny: /s/channel/postid are the ones of interest. |
|
02:56
π
|
JAA |
The other thing is only a "open in Telegram" page that doesn't show anything. |
|
02:56
π
|
Nikchemny |
JAA: Looks like that and looks like crap http://web.archive.org/web/20200630152730/https://t.me/MINSAPCuba/73 |
|
02:56
π
|
JAA |
That's not /s/. |
|
02:56
π
|
Nikchemny |
In AB collection, btw |
|
02:56
π
|
JAA |
https://web.archive.org/web/20200630154215/https://t.me/s/MINSAPCuba/73 |
|
02:57
π
|
Nikchemny |
Yes, sorry |
|
02:58
π
|
Nikchemny |
JAA: Btw, it shows 10 previous and 10 next posts. So AB saves the same post 20 times |
|
02:58
π
|
JAA |
Yep |
|
02:59
π
|
JAA |
No way to avoid that. |
|
02:59
π
|
JAA |
Looks like we're not saving these attachments(?): https://web.archive.org/web/20200630160216/https://t.me/s/MINSAPCuba/44 |
|
03:00
π
|
Nikchemny |
Yes, of course. Only Telegram users can do that |
|
03:00
π
|
JAA |
Doctissimo archival started. Retrieving 1 GB per minute, so I might run out of disk space quickly. |
|
03:01
π
|
JAA |
But it handles ~40 req/s just fine so far. |
|
03:02
π
|
Nikchemny |
Looks like Telegram is not too friendly for not registered people. Only pics and text. No files, no audios, no videos |
|
03:03
π
|
Nikchemny |
JAA: attachments can be saved only in app (or in web version, but I've never used it) |
|
03:04
π
|
JAA |
Nikchemny: Maybe there is a way to access them, but it's just hidden very well. |
|
03:05
π
|
Nikchemny |
Β―\_(γ)_/Β― |
|
03:06
π
|
JAA |
So disk space shouldn't be too much of an issue. Retrieving 1 GB per minute, but it compresses down to 120 MiB or so. :-) |
|
03:09
π
|
JAA |
Just to be clear: I'm only fetching the category and topic pages, nothing else. |
|
03:10
π
|
Nikchemny |
JAA: So, what does AT think about Telegram chats? Maybe just make bot that could join important chats and then just scroll them as long as possible? I know it can't be in WBM and (as I saw once) there is # in chat's link, so WBM can't show it |
|
03:10
π
|
Nikchemny |
*join in web version |
|
03:12
π
|
Nikchemny |
There is https://t.me/lurkisdead , chat of lurkmore.to |
|
03:13
π
|
PovAddict |
could use the telegram API as an actual registered user, but I'm not sure if Telegram servers would get angry and rate-limit (or ban), so I'd be reluctant to try with my regular account / phone number |
|
03:14
π
|
JAA |
I bet there's something in the terms about it, and they'd ban quickly when discovered or reported. |
|
03:14
π
|
PovAddict |
I don't know if bot accounts can read past chatgroup history |
|
03:15
π
|
Nikchemny |
Em, why? Do you think they can recognize when just person try to scroll chat and when bot scroll and saves it? |
|
03:16
π
|
Nikchemny |
Nope, I don't mean Telegram bots. Just use clear number, register with it and save as more as possible chat? |
|
03:19
π
|
JAA |
Doctissimo performance improved over the past 10 minutes. Maybe they're using autoscaling. I'm now at 64 req/s, and response times reduced by a third. |
|
03:19
π
|
Nikchemny |
JAA PovAddict: Is this a crazy idea? |
|
03:20
π
|
JAA |
No, but it will probably break quickly. |
|
03:22
π
|
Nikchemny |
Well, for the first try, I think, https://t.me/lurkisdead is good. |
|
03:25
π
|
Nikchemny |
JAA: Btw, anyone can download all his chats if he/she uses desktop app |
|
03:26
π
|
Nikchemny |
Maybe with attachments |
|
03:47
π
|
|
Raccoon has joined #archiveteam-bs |
|
03:58
π
|
|
qw3rty_ has quit IRC (Ping timeout: 622 seconds) |
|
04:13
π
|
Nikchemny |
. |
|
04:21
π
|
|
Nikchemny has quit IRC (https://mibbit.com Online IRC Client) |
|
05:10
π
|
|
wyatt8740 has quit IRC (Ping timeout: 260 seconds) |
|
05:12
π
|
|
wyatt8740 has joined #archiveteam-bs |
|
06:15
π
|
|
britmob has quit IRC (Read error: Connection reset by peer) |
|
06:30
π
|
|
qw3rty has joined #archiveteam-bs |
|
06:41
π
|
|
britmob has joined #archiveteam-bs |
|
07:32
π
|
|
jshoard has joined #archiveteam-bs |
|
07:33
π
|
|
PovAddict has quit IRC (Read error: Operation timed out) |
|
09:04
π
|
|
HP_Archiv has joined #archiveteam-bs |
|
10:11
π
|
|
lennier1 has quit IRC (Read error: Connection reset by peer) |
|
10:13
π
|
|
lennier1 has joined #archiveteam-bs |
|
10:54
π
|
JAA |
Doctissimo is nearly done, just a few huge topics remaining. |
|
10:54
π
|
JAA |
The SexualitΓ© section of Doctissimo* |
|
10:56
π
|
VoynichCr |
https://github.blog/2020-07-16-github-archive-program-the-journey-of-the-worlds-open-source-code-to-the-arctic/ |
|
10:57
π
|
VoynichCr |
"First, their well-known Wayback Machine is accessing and archiving raw GitHub data as WARCs, or Web ARChive files. As of this writing they have archived some 55TB of data. Second, they have the goal of making entire archived GitHub repositories available via βgit clone,β while also keeping repo comments, issues, and other metadata easily accessible on the web. This second initiative is well underway and initial archiving is expected |
|
11:16
π
|
JAA |
*record scratch* Yep, that's us, and you probably wonder how we got into this situation. |
|
11:16
π
|
JAA |
#gitgud on hackint |
|
15:04
π
|
|
BlueMax has quit IRC (Read error: Connection reset by peer) |
|
15:10
π
|
|
fredgido_ has quit IRC (Read error: Connection reset by peer) |
|
15:19
π
|
|
Arcorann has quit IRC (Read error: Connection reset by peer) |
|
15:31
π
|
|
systwi_ has joined #archiveteam-bs |
|
15:36
π
|
|
systwi has quit IRC (Read error: Operation timed out) |
|
16:00
π
|
|
kiska has quit IRC (Remote host closed the connection) |
|
16:01
π
|
|
kiska has joined #archiveteam-bs |
|
16:05
π
|
|
britmob has quit IRC (Read error: Operation timed out) |
|
16:43
π
|
|
schbirid has joined #archiveteam-bs |
|
16:45
π
|
|
britmob has joined #archiveteam-bs |
|
17:04
π
|
|
VerifiedJ has joined #archiveteam-bs |
|
17:17
π
|
|
Nikchemny has joined #archiveteam-bs |
|
17:18
π
|
Nikchemny |
JAA: There is https://docplayer.ru/ and there is example of the document: https://docplayer.ru/34254329-A-a-pleshakov-s-a-pleshakov-enciklopediya-puteshestviy-strany-mira-kniga-dlya-uchashchihsya-nachalnyh-klassov.html |
|
17:18
π
|
Nikchemny |
Can AB save PDF or not? |
|
17:21
π
|
Nikchemny |
Ah, here is link: https://docplayer.ru/storage/54/34254329/1595269256/uooJxGU_y6Zjzfubsxfreg/34254329.pdf . It can be reached only with Google-captcha |
|
17:32
π
|
Nikchemny |
JAA: Btw, I made https://archive.st/archive/2020/7/www.mk.ru/wae8/ , copy of https://www.mk.ru/politics/2020/07/20/kadyrov-otvetil-ssha-na-sankcii-fotografiey-s-avtomatami.html . Image ( https://archive.st/archive/2020/7/www.mk.ru/wae8/July202020508pm-58i1956pyt8zflwu0haadkhn4r7a9ecm.jpg ) looks similar, but archived copy is not |
|
17:32
π
|
Nikchemny |
https://archive.st/archive/2020/7/www.mk.ru/wae8/www.mk.ru/politics/2020/07/20/kadyrov-otvetil-ssha-na-sankcii-fotografiey-s-avtomatami.html |
|
17:32
π
|
|
PovAddict has joined #archiveteam-bs |
|
17:33
π
|
|
DogsRNice has joined #archiveteam-bs |
|
17:33
π
|
Nikchemny |
PovAddict: Which Telegram chat are you gonna to save? |
|
17:48
π
|
PovAddict |
I have no time for any of this |
|
18:05
π
|
|
prq has joined #archiveteam-bs |
|
18:23
π
|
|
Nikchemny has quit IRC (Quit: https://mibbit.com Online IRC Client) |
|
18:30
π
|
|
icedice has joined #archiveteam-bs |
|
18:39
π
|
SketchCow |
-- archive team slogan |
|
19:54
π
|
|
lennier2 has joined #archiveteam-bs |
|
19:55
π
|
|
lennier1 has quit IRC (Ping timeout: 260 seconds) |
|
19:55
π
|
|
lennier2 is now known as lennier1 |
|
20:00
π
|
|
Nikchemny has joined #archiveteam-bs |
|
20:02
π
|
Nikchemny |
katocala: Please add links from articles from this category https://ru.wikipedia.org/wiki/ΠΠ°ΡΠ΅Π³ΠΎΡΠΈΡ:ΠΡΠ³Π°Π½ΠΈΠ·Π°ΡΠΈΠΈ_Π²_Π ΠΎΡΡΠΈΠΈ,_ΡΠ°ΠΌΠΎΡΡΠΎΡΡΠ΅Π»ΡΠ½ΠΎ_ΠΏΡΠΈΡΡΠΆΠ΄Π°ΡΡΠΈΠ΅_ΡΡΡΠ½ΡΠ΅_ΡΡΠ΅ΠΏΠ΅Π½ΠΈ to https://www.archiveteam.org/index.php?title=ArchiveBot/Educational_institutions/list#Russia |
|
20:03
π
|
Nikchemny |
And this https://ru.wikipedia.org/wiki/%D0%9A%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F:%D0%A0%D0%BE%D1%81%D1%81%D0%B8%D0%B9%D1%81%D0%BA%D0%B8%D0%B5_%D0%B8%D0%BD%D1%81%D1%82%D0%B8%D1%82%D1%83%D1%82%D1%8B_%D0%B8%D1%81%D0%BA%D1%83%D1%81%D1%81%D1%82%D0%B2%D0%B0_%D0%B8_%D0%BA%D1%83%D0%BB%D1%8C%D1%82%D1%83%D1%80%D1%8B |
|
20:11
π
|
|
schbirid has quit IRC (Remote host closed the connection) |
|
20:15
π
|
|
Docti has joined #archiveteam-bs |
|
20:20
π
|
|
Docti has quit IRC (Ping timeout: 252 seconds) |
|
20:51
π
|
|
HP_Archiv has quit IRC (Quit: Leaving) |
|
20:52
π
|
paul2520 |
I saw this petition being circulated, with someone who has inside connections suggesting there have been layoffs and MNopedia.org is at risk. It's a fabulous, mostly Creative-Commons resource used on Wikipedia (some articles are heavily based on MNopedia articles). |
|
20:52
π
|
paul2520 |
I know at least some of the content is in the Wayback Machine... but can we get a crawl? |
|
20:53
π
|
paul2520 |
I'd be happy to run one myself (my bandwidth can support it), but it doesn't seem to be MediaWiki |
|
20:53
π
|
paul2520 |
oh sorry, petition link: https://docs.google.com/forms/d/e/1FAIpQLSdeqbOJ9-tmdrwXXfvbgR5BZASPh5jqZVkuyj5S-ox0VADQCQ/viewform |
|
20:54
π
|
paul2520 |
...in the meanwhile, I'll run a wget with mirror options today or overnight. |
|
21:14
π
|
|
Dallas has joined #archiveteam-bs |
|
21:16
π
|
Nikchemny |
paul2520: Why not just ask for saving https://www.mnopedia.org/ on AB channel? |
|
21:17
π
|
Nikchemny |
Or it's too big? |
|
21:18
π
|
paul2520 |
Nikchemny:ah, I always ask in the wrong #archive* channel |
|
21:18
π
|
Nikchemny |
#archivebot ? |
|
21:18
π
|
paul2520 |
thanks. |
|
21:18
π
|
paul2520 |
done |
|
21:18
π
|
paul2520 |
it's probably relatively small |
|
21:20
π
|
Nikchemny |
Btw, it looks like people on AB are very busy. I think (if they won't save it) you can ask for it again. |
|
21:20
π
|
Nikchemny |
paul2520: Btw, it is on mediawiki or not? |
|
21:21
π
|
paul2520 |
I don't think so |
|
21:21
π
|
paul2520 |
any ETA, like tomorrow? or more like give it a week and see if things are quieter/slower? |
|
21:21
π
|
paul2520 |
(or do you think asking at a weird time might work?) |
|
21:23
π
|
Nikchemny |
I think that your question could be sink in all this text. Sorry, I'm not at AB-channel, so I can't see what's happening there. |
|
21:26
π
|
Nikchemny |
paul2520 |
|
21:37
π
|
|
jshoard has quit IRC (Leaving) |
|
21:44
π
|
|
VerifiedJ has quit IRC (Quit: Leaving) |
|
21:50
π
|
|
Nikchemny has quit IRC (Quit: Page closed) |
|
22:16
π
|
|
Dallas has quit IRC (Quit: Dallas) |
|
22:17
π
|
|
Dallas has joined #archiveteam-bs |
|
22:17
π
|
|
HP_Archiv has joined #archiveteam-bs |
|
22:30
π
|
|
Jake has quit IRC (Read error: Connection reset by peer) |
|
22:30
π
|
|
Jake5 has joined #archiveteam-bs |
|
22:30
π
|
|
Jake5 is now known as Jake |
|
22:48
π
|
|
Dallas has quit IRC (Quit: Dallas) |
|
22:52
π
|
|
Dallas has joined #archiveteam-bs |
|
23:08
π
|
|
Dallas has quit IRC (Quit: Dallas) |
|
23:09
π
|
|
Dallas has joined #archiveteam-bs |
|
23:10
π
|
|
Arcorann has joined #archiveteam-bs |
|
23:53
π
|
|
BlueMax has joined #archiveteam-bs |
|
23:55
π
|
JAA |
The Doctissimo SexualitΓ© finished this afternoon. I'm considering grabbing all the other forums as well. It's fast and not too big, so why not? |
|
23:56
π
|
JAA |
Not too big in terms of WARC size, that is. The 54 million posts from SexualitΓ© were only 60-odd GiB. |
|
23:57
π
|
paul2520 |
nice |