Solid electrolytes provide a route to the development of all-solid-state batteries that can potentially surpass the safety and performance of conventional liquid electrolyte-based devices. Sulfide solid electrolytes have received particular attention as a result of their high ionic conductivities. One of the main reasons for such high ionic conductivity is the apparently reduced grain boundary resistance of sulfide solid electrolytes compared to their oxide counterparts, but this is not fully established. Using two model electrolyte systems, Na3PS4 and Na3PO4, we apply a novel microscale simulation approach to analyze ionic transport in polycrystalline materials with various grain volumes. For Na3PO4, high grain boundary resistance is found, with the Na-ion conductivity decreasing with decreasing grain volume. For Na3PS4, the overall influence of grain boundaries (GBs) is significantly reduced compared to the oxide. Detailed analysis reveals a minimal change in the local structures and Na-ion conduction mechanism between bulk and polycrystalline Na3PS4, whereas the change is far more substantial for Na3PO4, with evidence of over-coordination of Na ions at the GBs. Our microscale approach helps to explain the fundamentally different influences of GBs on ion transport in phosphate and thiophosphate solid electrolytes.