• rockSlayer@lemmy.world
        link
        fedilink
        arrow-up
        38
        ·
        7 months ago

        Well that’s part of the thing. Web scraping doesn’t get covered by policies. Like, they could ban your ip or any accounts you have, but web scraping itself will always be acceptable. It’s why projects like NewPipe and Invidious don’t care about YouTube cease and desist letters.

          • rockSlayer@lemmy.world
            link
            fedilink
            arrow-up
            3
            ·
            7 months ago

            Yea, I’ve seen those pop-ups when trying to find something out. It sucks but isn’t a significant barrier to web scraping

          • AeroLemming@lemm.ee
            link
            fedilink
            English
            arrow-up
            1
            ·
            7 months ago

            Doesn’t that only happen on the mobile version? Either way, it’s stupid and annoying. Google should start de-ranking sites that add barriers to content, but I know they never will.

            • werefreeatlast@lemmy.world
              link
              fedilink
              arrow-up
              1
              ·
              7 months ago

              I tried that on my desktop. So long as you are not actually logged in you cannot see the communities that are too small for a review or too adult after a review.

              • AeroLemming@lemm.ee
                link
                fedilink
                English
                arrow-up
                2
                ·
                7 months ago

                Ugh, what a fucking shitshow. I know it won’t happen quickly or easily, but I’m hoping to see more people on federated platforms in the next decade or two. It’s the only way for us to take the internet back from these greedy bastards.

        • AeroLemming@lemm.ee
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 months ago

          Is it any different for an “API”? I don’t think there’s a very big difference between an HTTP endpoint that returns HTML and an HTTP endpoint that returns JSON.

          • krippix@feddit.de
            link
            fedilink
            English
            arrow-up
            1
            ·
            7 months ago

            In what way?

            HTML definitely provides more overhead than json if you only care about the data.

            • AeroLemming@lemm.ee
              link
              fedilink
              English
              arrow-up
              1
              ·
              7 months ago

              Legally. OC stated that NewPipe doesn’t worry about legal threats because they scrape instead of using an official API.

          • folkrav@lemmy.ca
            link
            fedilink
            arrow-up
            1
            ·
            7 months ago

            Parsing absolutely comes with a lot more overhead. Especially since many websites integrate a lot of JS interactivity nowadays, you oftentimes don’t get the full contents you’re looking for straight out of the HTML you’re getting out of your HTTP request, depending on the site.

      • freebread@lemm.ee
        link
        fedilink
        English
        arrow-up
        9
        ·
        7 months ago

        Still waiting for the news that they took down old.reddit. Without the third party apps, that was the only way it could still be usable.

    • IphtashuFitz@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      7 months ago

      We use Akamai where I work for security, CDN, etc. Their services make it largely trivial to identify traffic from bots. They can classify requests in real time as coming from known bots like Googlebot to programming frameworks like python & java to bots that impersonate Googlebot, to virtually any other automated traffic from unknown bots.

      If Reddit was smart they’d leverage something like that to allow Google, Bing, etc. to crawl their data and block all others, or poison others with bogus data. But we’re talking about Reddit here…

  • ArugulaZ@kbin.social
    link
    fedilink
    arrow-up
    41
    ·
    7 months ago

    Sure, why not. People gave you all the information on Reddit for free, you might as well sell it to the highest bidder without compensating them. I call it the “Veasey maneuver.”

  • SupraMario@lemmy.world
    link
    fedilink
    arrow-up
    17
    ·
    7 months ago

    This makes me wonder if they sell nsfw subs data…if they don’t mods could just make their subs nsfw and the data doesn’t get collected…might be the same for accouts set to NSFW as well.

      • Zahille7@lemmy.world
        link
        fedilink
        arrow-up
        2
        ·
        7 months ago

        There are still plenty of subs that are considered 18+ when they technically shouldn’t have to be.

    • Boozilla@lemmy.world
      link
      fedilink
      English
      arrow-up
      13
      ·
      7 months ago

      Could be a good tactic. But spez and his goons tend to crack down on mods that do things they don’t like. Especially things that could impact revenue.

    • LustyArgonian@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      8
      ·
      7 months ago

      I’m actually pretty certain the NSFW/porn subreddits are involved in something nefarious, possibly human-trafficking adjacent. u/spez himself was a moderator of r/jailbait. When caught, he made it so subreddits can hide who mods them. They also regularly delete NSFW/porn subreddits that aren’t the mainstream ones (they deleted mine).

      To be approved to post on the mainstream porn reddits, you have to be verified, usually by taking a pic with a sign containing date, username, and subreddit. It’s unknown who reviews this (no mod transparency) or how many images reddit has collected over the years. Reddit, unlike onlyfans, also doesn’t verify age during this process.

      Posts that are approved tend to be suspicious - it’s clear these girls are working closely with the mods to curate certain images. Pokimane for example, or other egirls, will get approved for posts that don’t follow sub guidelines or will be pinned to the top. It’s just interesting which girls with an onlyfans are allowed to post, and which girls receive permabans.

      It’s almost like they are forcing these girls into dating them so they can post on their subs, and they block any competition for their girls. They’re porn pimps and spying on us.

  • demizerone@lemmy.world
    link
    fedilink
    English
    arrow-up
    6
    ·
    7 months ago

    I just signed up for lemmy.world today. Last week I got my first account suspension in 15 years because some automated tool didn’t like my joke about the Boeing Starliner and how the Astronauts should be concerned regarding the recent history of Boeing. Reason for suspension was for making threats. It was more like threats to Boeing’s reputation. Deleted all my top comments and closed the account. Reddit is dead.