Cybersecurity firm Crowdstrike pushed an update that caused millions of Windows computers to enter recovery mode, triggering the blue screen of death. Learn …

  • JeeBaiChow@lemmy.world
    link
    fedilink
    English
    arrow-up
    75
    arrow-down
    12
    ·
    edit-2
    4 months ago

    Whoda thunk automatic updates to critical infrastructure was a good idea? Just hope healthcare life support was not affected.

    • Toribor@corndog.social
      link
      fedilink
      English
      arrow-up
      69
      arrow-down
      1
      ·
      4 months ago

      Many compliance frameworks require security utilities to receive automatic updates. It’s pretty essential for effective endpoint protection considering how fast new threats spread.

      The problem is not the automated update, it’s why it wasn’t caught in testing and how the update managed to break the entire OS.

      • Joe@discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        8
        arrow-down
        1
        ·
        edit-2
        4 months ago

        It is pretty easy to imagine separate streams of updates that affect each other negatively.

        CrowdStrike does its own 0-day updates, Microsoft does its own 0-day updates. There is probably limited if any testing at that critical intersection.

        If Microsoft 100% controlled the release stream, otoh, there’d be a much better chance to have caught it. The responsibility would probably lie with MS in such a case.

        (edit: not saying that this is what happened, hence the conditionals)

        • Toribor@corndog.social
          link
          fedilink
          English
          arrow-up
          13
          ·
          4 months ago

          I don’t think that is what happened here in this situation though, I think the issue was caused exclusively by a Crowdstrike update but I haven’t read anything official that really breaks this down.

          • barsquid@lemmy.world
            link
            fedilink
            English
            arrow-up
            15
            ·
            4 months ago

            Some comments yesterday were claiming the offending file was several kb of just 0s. All signs are pointing to a massive fuckup from an individual company.

            • Wiz@midwest.social
              link
              fedilink
              English
              arrow-up
              5
              arrow-down
              1
              ·
              4 months ago

              Which makes me wonder, did the company even test it at all on their own machines first?

      • LainTrain@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        8
        arrow-down
        26
        ·
        4 months ago

        Nah EDR is pointless like all of cybersecurity. All these compliance frameworks are just a further grift to get a slice of B2B procurement budgets. The practice of cybersecurity has caused a more severe widespread outage than any malware ever could.

          • fishpen0@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            4
            ·
            4 months ago

            OP is not entirely wrong. At least in Linux land you can now implement EDR like functionality entirely with EBPF without installing a fucking rootkit. So traditional EDR products are a grift if you are on the bleeding edge.

        • jumjummy@lemmy.world
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          2
          ·
          edit-2
          4 months ago

          Ok Russian comrade. Security in companies is terrible. You’re right. It’s just a giant grift.

          Now, go buy some limited time offer fight fight fight shoes from agent orange.

    • Kairos@lemmy.today
      link
      fedilink
      English
      arrow-up
      21
      arrow-down
      3
      ·
      4 months ago

      Hospital stuff was affected. Most engineers are smart enough to not connect critical equipment to the Internet, though.

      • Dr. Arun Wadhwa@lemmy.world
        link
        fedilink
        English
        arrow-up
        20
        ·
        4 months ago

        I’m not in the US, but my other medical peers who are mentioned that EPIC (the software most hospitals use to manage patient records) was not affected, but Dragon (the software by Nuance that we doctors use for dictation so we don’t have to type notes) was down. Someone I know complained that they had to “type notes like a medieval peasant.” But I’m glad that the critical infrastructure was up and running. At my former hospital, we used to always maintain physical records simultaneously for all our current inpatients that only the medical team responsible for those specific patients had access to just to be on the safe side.

        • JeeBaiChow@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          ·
          4 months ago

          That’s actually a very smart idea, keeping physical records of every inpatient. Wonder why the ai companies don’t do transcription of medical notes, instead of trying to add ai features to my washer/ dryer combo. Just seems like a very practical use of the tech

          • deranger@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            3
            ·
            edit-2
            4 months ago

            Wonder why the ai companies don’t do transcription of medical notes

            They do, one of the things my hospital is working on implementing, much to my chagrin, is an AI thing where the doctor leaves their phone out during the visit. It listens to the patient and the doctor and generates a note. I think it’s a Nuance product, I’m not directly involved with the implementation.

            For me, as soon as I see a doctor have his phone out I’m telling him to put that shit away and I don’t consent to some app listening to what I’ve got to say.

            • JeeBaiChow@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              4 months ago

              I meant to transcribe handwritten patient notes to dogital after the patient is released. That way op can retain the physical records for inpatients to mitigate another crowdstrike situation, and still have searchable records long term. Recording a patient consultation sounds like a bad idea all round. They’d have to figure out how to read doctors handwriting though,bbut I gather most of the difficulty is due to the cryptic names they have for things.

            • conciselyverbose@sh.itjust.works
              link
              fedilink
              English
              arrow-up
              1
              ·
              4 months ago

              I doubt you can decline, any more than you can tell them not to use a notebook.

              It is subject to actually serious HIPAA regulations though.

              • deranger@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                1
                ·
                4 months ago

                I’m aware of the regulations, I’m an Epic analyst and been in the world of healthcare since 2009 (used to be an MLS). Watch me decline right out the door and immediately contact patient relations. Put your fucking phone away. Healthcare has managed for quite a while without them.

                Saying a personal cell phone is equivalent to a written notebook is wild. Can you compromise a notebook with malware?

                • conciselyverbose@sh.itjust.works
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  ·
                  4 months ago

                  The doctor will very likely have no choice but to send you to complain to someone else. Not recording will violate policy if that’s what their system is, and it’s an entirely reasonable policy to have. Medical interactions are heavily documented for a reason, and it’s because mistakes can literally kill people.

                  All your records are already electronic, and a notebook is a far bigger security risk than a cell phone is. It’s not encrypted. Anyone can walk away with it.

                  • deranger@sh.itjust.works
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    ·
                    4 months ago

                    I disagree. I don’t consent to a doctor having his personal device out recording. Doesn’t seem like either of us will be changing our minds, so I’ll bid you adieu.

        • RunningInRVA@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          4 months ago

          This is pretty much correct. I work in an Epic shop and we had about 150 servers to remediate and some number of workstations (I’m not sure how many). While Epic make not have been impacted, it is a highly integrated system and when things are failing around it then it can have an impact on care delivery. For example if a provider places a stat lab order in Epic, that lab order gets transmitted to an integration middleware which then routes it to the lab system. If the integration middleware or the lab system are down, then the provider has no idea the stat order went into a black hole.

          • deranger@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            3
            ·
            4 months ago

            Our lab was absolutely fucked from multiple integrations going down. I’m a Cupid analyst and we weren’t really affected. What app do you work on?

            • RunningInRVA@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              4 months ago

              I’m an integration guy at my roots but I lead a variety of different teams at the moment. We use Corepoint as one of our interface engines and it shat the bed big time. We had to restore it from backup, which was nuts in my opinion. We had a variety of apps impacted.

              • deranger@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                2
                ·
                4 months ago

                That’s cool. I was going to move over to our integration team but I’m looking into Epic consulting instead. Our integration team was very busy on Friday along with our clinical apps team. We use Cloverleaf for our interface engine, I’ve got a bit of experience poking around in there. HL7 is interesting, but I’d like to learn FHIR. Do you have a Bridges cert?

                • RunningInRVA@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  ·
                  4 months ago

                  I’m Bridges certified as well as in Cloverleaf, which we also use. FHIR is great but it doesn’t require much in the way of integration engineers.

        • deranger@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          4
          ·
          4 months ago

          I’m an Epic analyst - while Epic was fine, many of our third party integrations shit the bed. Cardiology (where I work) was mostly unaffected aside from Omnicell being down, but the laboratory was massively fucked due to all the integrations they have. Multiple teams were quite busy, I just got to talk to them about it eventually.

        • datendefekt@lemmy.ml
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          2
          ·
          4 months ago

          “type notes like a medieval peasant.”

          Huh. I thought medieval peasants were usually illiterate? Even less computer literate?

    • Juvyn00b@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      4 months ago

      I work healthcare adjacent and some providers were affected as expected. Hoping as well that those critical systems were not, but that chance is non zero.