You can use swift package manager to create an executable target, which you can then run on macOS from the command line. Most frameworks (aside from UI) are shared between iOS and macOS like AVFoundation. If you don’t have access to macOS, you can still use swift package manager to build and run code on Linux or Windows, though you unfortunately won’t have access to any Apple frameworks.

When Apple introduced the iPhone they required automatic reference counting to be used in Objective-C rather than tracing garbage collection (the language supported both) due to performance reasons (iPhones were significantly slower than Macs). At least Apple seems to think that reference counting is faster than tracing garbage collectors. The compiler can do a lot to remove unnecessary releases and retains. Additionally each retain is just incrementing an integer, and each release is just decrementing an integer and comparing the integer to 0, so the overhead is pretty small.

Hacking with Swift is a great resource. They have quite a few books related to development in Apple platforms (some free). You’ll probably want to do some research on if you want to learn SwiftUI or UIKit. UIKit is still used much more frequently in industry, but SwiftUI is becoming more mainstream, and is easier to learn. Hacking with Swift has books for both frameworks.

Personally, I’d recommend just jumping into writing a simple app (probably by following along with a tutorial) since you already know how to code, and have a grasp of the basics of Swift. There isn’t too much you need to learn about Xcode initially, and any tutorial for creating your first app should walk you through any Xcode specific steps you do need to do. Most of your learning will probably be related to either UIKit or SwiftUI, which are the frameworks you use for actually getting stuff on the screen. While learning these, you’ll undoubtedly pick up more knowledge about Swift and Xcode, which will improve those skills. I certainly don’t think you need to become an expert in Swift or Xcode before learning UIKit or SwiftUI. As you learn more, you’ll learn what areas you need to improve on, and can tailor your journey from there.

Quartz is a layer beneath SwiftUI or AppKit. SwiftUI is still using Quartz under the hood. The way you use Quartz directly from SwiftUI vs AppKit is a bit different, though still fairly similar. A more fair comparison of the SwiftUI code would be:

struct HelloWorldView: View {
  var body: some View {
    Canvas { context, _ in
      context.draw(
        Text("HelloWorld")
          .font(.system(size: 24))
          .foregroundColor(.black),
        at: CGPoint(x: 20, y: 20)
      )
    }
  }
}

Alternatively an AppKit solution (not using Quartz directly) would be something like:

class HelloWorldView: NSView {
  override init(frame frameRect: NSRect) {
    super.init(frame: frameRect)
    let text = NSAttributedString(
      string: “Hello World”,
      attributes: [.font: NSFont.systemFont(ofSize: 24), .foregroundColor: NSColor.black]
    )
    let label = NSTextField(labelWithAttributedString: text)
    addSubview(label)
  }

  required init?(coder: NSCoder) {
    fatalError()
  }
}

In either AppKit or SwiftUI, you can access Quartz directly to implement custom views. However, most of the time the UI code you write in either SwiftUI or AppKit won’t call Quartz directly at all, but will instead be composed of built-in views, like (NS)TextField or (NS)Button. Under the hood, SwiftUI is mainly just using the AppKit components at the moment, but provides a significantly nicer way to use them (especially in regards to layout and data synchronization).