Can you Give any Suggestion?

I think you might want to use 'nofollow' attributes for your new added links.
Generally speaking, links with your keywords in anchor text would help your sites in ranking but not hurt.

I agree with mate if you use nofollow attribute for your site than it will help you more.

You question is little incomplete. Are you talking about PR ranking or SERPs ranking. And How did you know it was your links that are taking your rank down?

yes nofollow tag is great way to stop google crawl that hyperlink.

Agree with Airforce1. Using nofollow attrtibute you can stop google to crawl that hyperlinked site.

Your question is somewhat vague. If I got it right, you are talking about outgoing links. Just use external no follow. And one more thing just giving one or two links back from your sites does not have any serious impact on your website's ranking.

Using hyperlinks helps in increasing you website ranking. I don't think this the reason behind decreasing of your website position

If the hyperlink not very important to your site,why not remove it ?

Try to apply no-follow attribute to the link.