Hi,
"Allow All application" rule may be the rule which is allowing the RDP traffic.
Create a rule --> Block connection for traffic which has a destination of port 3389.
Move this rule to the top and check if we still face the same issue.
Let me know if you face any issues while creating the firewall rule.