>>> We have changed the Polling interval time to 2 second but still we are facing the performance issue,
To process a file of size 1037B it is taking 13- 15 seconds.
Are you doing content conversion(FCC) in your sender channel?
By any chance, Did you enable virus scan for your scenario?
>>>Also, is there any way where in we can specify the number of files to be picked & processed in a single Poll.
There is no such option. All the matching files would be picked and processed. Alternative solution is already suggested by Inaki.