Canonfire! Forums-viewtopic-Announcement: Search through Dragon Magazine and others

Last edited by lamashtu on Mon Jun 10, 2019 8:53 am; edited 1 time in total

Now with a web (and mobile-friendly) interface.

Instantly search through Dragon's full run of issues including The Strategic Review, the Best of issues, and the Annuals; Oerth Journal; the LGG; and whatever else we might cram into the database.

No ads. No frills.

Looking for that article that had something about orgies you vaguely remember from when you were 10? This can help you find it.

http://wait-wtf.com:3001

Updated:

As an experiment I've change the nature of searches over 4 characters in length to attempt to capture results that were being excluded due to bad OCR. An example of the problem being solved can be found here: http://wait-wtf.com:3001/terrible (page 21 of Dragon 10)

If there's any feedback on the change (false positives, slow results returns, etc) feel free to let me know.

Mon May 13, 2019 4:37 am

This is terrific, thank you!
Dave Guerrieri

Tue May 14, 2019 4:22 pm

What is the URL?

Thu May 16, 2019 3:17 am Re: Boccob Online

Thu May 16, 2019 7:04 am

That's correct Icarus, the backend is (mostly) the same as what boccob was using in IRC.

For you and anyone following the thread, and as I say in the FAQ, I can add more sources (Dungeon? Modules?), so everyone is welcome to let me know what they'd like to see.

Future development is contingent on use and feedback.

Fri May 17, 2019 3:12 am Re: Boccob Online

Mon Jun 03, 2019 6:23 pm

Any chance we could get the 1e rulebooks added into the mix too? :D

Allan.
_________________
Allan Grohe<br />https://www.greyhawkonline.com/grodog/greyhawk.html<br />https://grodog.blogspot.com/

Mon Jun 03, 2019 9:32 pm

Just testing it out... I did a search for "Iuz." Lots of hits, but nothing going back prior to Dragon #103. Is this due to OCR problems with early issues?
_________________
- Moderator/Admin (in some areas)/Member -

Tue Jun 04, 2019 5:48 pm

This is pretty cool, even with the limitations that Cebrion mentions. Happy

Fri Jun 07, 2019 10:31 am

Fri Jun 07, 2019 11:57 am

Fri Jun 07, 2019 2:55 pm

...and whatever, figured I'd improve the sorting a bit. It's still... weird in a way that isn't yet making sense to me. But I think you'll find it improved.

Fri Jun 07, 2019 6:44 pm

Yes, it would be great if it sorted sources numerically/alphabetically.

But...getting stuff to appear at all is at the least very useful.

The effort is greatly appreciated. Cool

_________________
- Moderator/Admin (in some areas)/Member -

Fri Jun 07, 2019 7:31 pm

Sat Jun 08, 2019 5:50 pm

Just tried it out, and it looks like it is working really well! Cool

_________________
- Moderator/Admin (in some areas)/Member -

Mon Jun 10, 2019 9:05 am

As an experiment I've change the nature of searches over 4 characters in length to attempt to capture results that were being excluded due to bad OCR.

An example of the problem being solved can be found here: http://wait-wtf.com:3001/terrible (page 21 of Dragon 10)

If there's any feedback on the change (false positives, slow results returns, etc) feel free to let me know.

Thu Jun 20, 2019 3:22 pm

After a some research, I implemented a new processing chain for OCRing the sources. Took about 200 hours of every core running flat out, but the result should be a reduction in false-negatives of around 10-15%, without any slowdown to the end user.

Future plans include:

- adding additional sources (including the long absent Dragons 147-149)

- providing an interface to filter out sources the user doesn't want to see

- allowing 'match' queries, which will permit searches that require more flexibility

Thu Jun 20, 2019 5:43 pm

+1 for OJ#15. I've used Magna Alumnae a bit IMC.

OJ#7 also has a bardic order of "Keepers" devoted to Wee Jas described in her article.

The flavor text in the Complete Bard's Handbook has some light Greyhawk references.

Wed Sep 04, 2019 8:18 am Lauding the Textual Search website.

Just for the sake of saying:
Lamashtu, this is an incredibly useful resource.
I've just recently used it in research for a "constellations" article for OJ30, coming out this month.
Andy Miller is writing a follow-up to his article in OJ22, and starting a web-series on Greyhawkonline.com, and it's resources like this that make research manageable when grinding through books and magazine for information.

What's more, in writing the introduction to the article series, I've repeated the search more than once, and recorded the results. I note that now, the results are all listed in order, (Dragon first, then the OJ, then other products), and there are more/better results for it.

Thank you, again! You've given a lot to the community!
_________________
Owner and Lead Admin: https://greyhawkonline.com<div>Editor-in-Chief of the Oerth Journal: https://greyhawkonline.com/oerthjournal</div><div>Visit my professional art gallery: https://wkristophnolen.daportfolio.com</div>

Sun Oct 06, 2019 5:39 pm

Good hear it was useful, makes me super happy to see a real use case!

With a new job, and more or less leaving the world of gaming again, I haven't worked on this much recently. I'm aware that there are some speed and out-of-order operations problems, but it's just not a super high priority right now.

I continue to support it and pay for its presence on the net, and that won't change.

Anyone who is trying to utilize it for a project and encounters a problem should probably email me (the FAQ page has my contact info), as my visits to Canonfire are very rare now.

Icarus in particular since you found it useful, please don't hesitate to reach out if there are additional books you'd like to see added. I still have many essentially ready to be added if I find the time or someone pokes me.

Sun Oct 06, 2019 7:15 pm

So I just did a search for Greyhawk. You know, just to see what would turn up.

Shocked

Horned Society next.

Sat Jun 13, 2020 10:34 am Database is down

Dear lamashtu,

I'm incredible sorry for this necromancy, but the collection is too valuable to let it vanish.
It seems like the link you posted does not work anymore. Strangly enough, your mirror of the Greyhawk Wiki still does its job.

I tried to commune with you via private messaging, alas, it seems like my spell failed and never reached out to you!
If you read this, please, bring back the searchable magic.

Greetings from Germany,
-r
_________________
There's never enough time to do all the nothing you want...

Tue Jan 04, 2022 7:53 pm

To further necro, this service is working again and reachable at https://wait-wtf.com:3002

I'm considering returning it to development.

Wed Jan 05, 2022 1:15 pm

oops. habits are strong.

that's http://wait-wtf.com:3002

https is not currently supported (or necessary).

Fri Jan 07, 2022 5:23 pm

Just checked this out for the first time today... what an incredible resource! I'll add my voice to the chorus of "thanks, Lamashtu!"