To fetch huge amount of data from sql server database you need to follow these steps:
i) Firstly you need to use the Standard way using ADO.net to fetch results. Because LINQ to SQL is ultimately an ORM (additional layer) so obviously it will lower the performance which is noticeable in just like you case. see here
ii) Then after using ADO Please review your table design if you have not missed anything like adding index,selecting wrong data type etc.
iii) Then Improve Stored Procedure performance
iv) To solve a huge amount of data divide Your Data.: Move your old data to another table if it is read only(or no longer going to be used) as you are telling that a minimum of 2000 rows is added daily. So your table size will be decreasing significantly. Please decide this decision very carefully.