Yeah, that was the idea. But, as with any model, it is a simplified representation of reality. It hides lots of details and complexity. The exact waveshapes, timing of the envelope segments, audio signal and modulation levels, digital resolution, samplerates, filter characteristics. All these factors determine the distinctive character of the synthesizer.
Some might remember the C64 game Impossible Mission. It featured speech and sample playback. Although it might not even deserve the qualification Lo-Fi. Nevertheless, it was like witnessing dark magic in those days.
This feature was not by design. The SID chip itself had a quirk that was exploited to play samples. It clicked when you turned the volume on or off. This could be used to play sampled sounds (as pwm/bitstreams). Similar to how in older PC’s the (on/off) piezo tweeter/speaker, that was intended to only produce an occasional beep, was exploited to play samples.
About a year ago I described how ADC/DAC using PWM works and how to recreate it in a VCV Rack patch. I was quite surprised it was able to pretty truthfully recreate a processed input sample. Because for the technique to work properly, it requires high samplerates.