Publicly searchable? - General

Welcome to the Functional Programming Zulip Chat Archive. You can join the chat here.

Sridhar Ratnakumar

Are the chat logs publicly searchable? If not can it be made so? Google juice is delicious.

TheMatten

You mean you want to search history through Google?

TheMatten

https://zulipchat.com/help/search-for-messages
Integrated search, being "domain-specific", seems to have some nifty features

Zulip has a powerful search engine under its hood. Search for messages using the search bar at the top of the screen. | Searches for messages in #design that have a link, that you starred, and that have the words new and logo. | The permalink for that search (web only) will look something like https://your-zulip-url/#narrow/stream/123-design/has/link/is/starred/search/new.20logo. | As you start typing, Zulip will suggest possible operator completions. Operators can be used with keywords, or on their own. For example,
TheMatten

@Sridhar Ratnakumar

Sridhar Ratnakumar

I mean publicly searchable on Google. You know, for discoverability. Esp. useful in cases when people search Google for things like 'how do I do foo in haskell' etc

TheMatten

Oh, yeah, that would be cool, not sure

Sridhar Ratnakumar

Are our streams private or public? cf. https://zulipchat.com/help/change-the-privacy-of-a-stream

Streams can be public or private, and private streams can have shared or protected history. See stream permissions for details on stream privacy settings. | As an organization administrator, you can always make a public stream private, but you must be subscribed to a private stream in order to make it public. | We're here to help! Email us at support@zulipchat.com with questions, feedback, or feature requests.
Sridhar Ratnakumar

Okay n/m - this doesn't look like the same thing

James King

https://github.com/leanprover-community/leanprover-community.github.io/blob/master/import.py is used to do the import and run by a scheduler, and the zulip team have create a standalone version from it: https://github.com/zulip/zulip-archive

Contribute to leanprover-community/leanprover-community.github.io development by creating an account on GitHub.
A tool for archiving and displaying Zulip chat channels. - zulip/zulip-archive
TheMatten

We could use this too - @Leonardo Taglialegne are you interested? :slight_smile:

Leonardo Taglialegne

Yeah, I think I can arrange that =)
Do we have a domain?

Sridhar Ratnakumar

If I write a log archiver in Haskell, would you folks be interested? :-D

Sridhar Ratnakumar

(I did the same for Slack: https://github.com/srid/Taut )

An ambitious Slack JSON export viewer. Contribute to srid/Taut development by creating an account on GitHub.
TheMatten

@Leonardo Taglialegne If we don't need nice TLD, we can get free domain at freenom.com

James King

Has anyone moved on this? Maybe we could host on github pages?

Sridhar Ratnakumar

I'll take a look at this today.

James King

Cool. No rush, I was going to offer to look into it if nobody else was interested. Cheers!

Sridhar Ratnakumar

I was thinking, static HTML archives can be added to keybase pages ( https://keybase.io/docs/kbp ) ... starting with zulip-archive (possibly to be replaced in future by a nicer Haskell based generator).

Sridhar Ratnakumar

The python archiver is utterly broken.

Sridhar Ratnakumar

Not totally convinced about keep using this, though.

Sridhar Ratnakumar

Had to fix a few things in the python code.

Sridhar Ratnakumar

the workflow is terrible, and code is badly written

Sridhar Ratnakumar

@James King If you'd like to take over the zulip-archive bot maintenance I can had over control of the github organization to you. This archiver is hardcoded to use github jekyll

James King

I'm agentultra on Github

TheMatten

@Sridhar Ratnakumar Wow, I can imagine using this - maybe we could find someone interested in writing this e.g. in Haskell as a cool project

Sridhar Ratnakumar

@James King Invited you to https://github.com/funprogzulip/funprogzulip.github.io

Contribute to funprogzulip/funprogzulip.github.io development by creating an account on GitHub.
Sridhar Ratnakumar

@TheMatten Yea, the archive idea is up for grabs, but I hope whoever picks it up will use my library for archive generation: https://github.com/srid/rib (I'm most happy to help!)

Right now, my hobby project is to play with reflex and svg, involving hexgrid layouts for rendering triominos (and later write AI for it)

Haskell library for writing your own static site generator - srid/rib
Sridhar Ratnakumar

Especially if they also gonna use nix, I can pair a bit in the project.

Ryan

FWIW if anyone is curious, here's the overarching tracking issue for this https://github.com/zulip/zulip/issues/13172

This is the master issue for the large project of supporting viewing logged-out Zulip history. This is useful for two major uses case: For community and open source projects, to make it possible to...
James King

@Sridhar Ratnakumar when generating the archive do you run the script and add the output to the repo? or are you using a different generator? (would it be useful to include the generator script in the repo?)

James King

I have some spare time to set this up on my server to automate the archive generation

Sridhar Ratnakumar

IIRC I was git add'ing the generated files manually.

James King

Ah ok. Should be pretty easy to script then. :thank_you:

Sridhar Ratnakumar

@James King I just wanted to let you know that I've been working on a Zulip archiver in Haskell. I'm at the point where I have pulled channels and streams, now need to sync messages as well.

Sridhar Ratnakumar

I think having publicly available logs would be a great distinctive feature compared to Slack; and it might get us more mindshare, and thus more users. :-) That's the motivation behind it.

Sridhar Ratnakumar

The repo is still private; but I can open it up (before ready milestone) if people are curious or want to contribute.

Pavan Rikhi

I'd be into checking it out if that's cool w/ you

Pavan Rikhi

I want more people here :D

Sridhar Ratnakumar

Okay, published here: https://github.com/srid/zulip-archive (work in progress)

WIP Zulip Archive viewer (statically generated HTML) - srid/zulip-archive
Sridhar Ratnakumar

Made some good progress on this. I like how this is going to have clean URLs for every topic, example /polysemy/monadio-instance-for-sem.html. Much Google juice to be provided.

Sridhar Ratnakumar

Public announcement - zulip archives are now publicly available at https://funprog.srid.ca/

Sridhar Ratnakumar

One less reason to stay at Slack :-D

TheMatten

BTW, you can take branding from Zulip to make it more "personalized" :slight_smile: (or I can send you original images of logo and icon)

Sridhar Ratnakumar

Ya, feel free to open up issues in https://github.com/srid/zulip-archive

WIP Zulip Archive viewer (statically generated HTML) - srid/zulip-archive
TheMatten

It's so cool - you even did links and code blocks! :heart:

Sridhar Ratnakumar

That's thanks to the zulip API which returns the raw HTML, so I didn't have to do all the rendering work!

Sridhar Ratnakumar

Uploaded files is the only area where their API lacks, as we have to do that manually (no API to fetch all uploaded files)

Sridhar Ratnakumar

Let's strategically inject URLs to specific conversation from the archive in fpslack as a form of advertising :D

Sridhar Ratnakumar

In latest master of zulip-archive, things should no longer be hardcoded. So if any of you guys want to play with it, or contribute towards implementing some of the features reported in Issues, feel free to take a look! @TheMatten @Pavan Rikhi ... I'm gonna focus on client work for a while

James King

That's dope! Nice work!

Sridhar Ratnakumar

It will be better if someone implements Pavan's idea https://github.com/srid/zulip-archive/issues/11

This would help display posts as rich results in search result pages: https://developers.google.com/search/docs/guides/intro-structured-data https://schema.org/docs/schemas.html The few relevant ty...
Sridhar Ratnakumar

I was searching Google for something and then our archive came in the results! image.png

Sridhar Ratnakumar

Did a google search for ipfs zettelkasten and this was the result! image.png

Sridhar Ratnakumar

That's your comment @Sam Stites

Sam Stites

: ) nice! Thanks for hosting the archives!

Pavan Rikhi

Imagine what it'll be like a year from now :grinning:

Sridhar Ratnakumar

I feel like ... we should 'brand' it. I don't like non-brand-y domains like funprog.srid.ca. Perhaps we can use haskell.zettel.page ... chat.haskell.zettel.page?

Sridhar Ratnakumar

With the idea being: chat archives, and wiki-esque content in one place ... our community Haskell site.

TheMatten

I guess something like that would make sense - though this Zulip instance is about FP in general, not just Haskell, so we should probably come up with something less specific

Sridhar Ratnakumar

though this Zulip instance is about FP in general, not just Haskell

lol I overlooked all that.