Best guess explanation for the 28% is that they have not been on or are not responding to the NS. Try creating a task to run inventory and using one of these 28% as a test machine.
Best guess for the 4% is that no inventory plug-in is installed. Be sure your target only includes those with the inventory plug-in.
Without knowing more that's where I'd start. If you have a client who misbehaves, agent logs would be beneficial. You can view them yourself by typing running regsvr32 to register the AexAgentDiagnostics.dll.