The CPU and Network limits are only soft requirements. You can specify relative measure in it and Squarescale will try to schedule services on machines that are not overloaded.
The memory limit however is strict and your service will not be able to go beyond that limit. If it tries to allocate more memory the Linux kernel will kill your container and it will be restarted. To determine the correct value for the amount of memory you allocate to your service you should benchmark your service and determine the highest amount of memory it will take, then add a security margin.
Under the hood we are using the nomad scheduler and their model for resource limits.