DuckDuckGo, Bing, Mojeek, and other search engines are not returning full Reddit results any more.

  • Cyborganism@lemmy.ca
    link
    fedilink
    English
    arrow-up
    23
    arrow-down
    2
    ·
    4 months ago

    Could it be possible to have one major global instance that aggregates everything so it can be indexed by search engines? Would that work? Or do I not fully understand how federation works?

    • wholookshere@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      31
      arrow-down
      2
      ·
      4 months ago

      That would defeat the purpose of federation.

      It becomes a central choke point of moderation. Who gets to decide what instances are part of global and which ones aren’t. Because a free for all isn’t going to end well. And then you’re back at Reddit.

      • WanderingVentra@lemm.ee
        link
        fedilink
        English
        arrow-up
        12
        ·
        edit-2
        4 months ago

        I wonder if you could have an instance federated to every other instance just for archived purposes, to save the data on every other instance’s post and comment. Because copies of posts and comments are saved to federated instances, too, right? Or do I understand the tech wrong?

        So it could have an admin team but no users, to prevent people worried about spammers and bots joining that instance to get around defederation rules. Maybe it just has a bot that crawls Lemmy, looking for instances to federate to. Could that work?

      • rbits@lemm.ee
        link
        fedilink
        English
        arrow-up
        2
        ·
        4 months ago

        Right, but having a centralised search index thingy is better than none at all. Maybe there could be something where it’s a joint effort from admins from many of the biggest servers, idk if that would work.

    • barsoap@lemm.ee
      link
      fedilink
      English
      arrow-up
      5
      ·
      edit-2
      4 months ago

      Lemmy search already is quite excellent… at least here on lemm.ee, we don’t have many communities but tons of users subscribed to probably about everything on the lemmyverse so the servers have it all.

      It might be interesting to team up with something like YaCy: Instances could operate as YaCy peers for everything they have. That is, integrate a p2p search protocol into ActivityPub itself so that also smaller instances can find everything. Ordinary YaCy instances, doing mostly web crawling, can in turn use posts here as interesting starting points.