Welcome to My Blog

Indexing as many pages on your website as possible can be very tempting for marketers who are trying to boost their search engine authority.

But, while it’s true that publishing more pages that are relevant for a particular keyword (assuming they’re also high quality) will improve your ranking for that keyword, sometimes there’s actually more value in keeping certain pages on your website out of a search engine’s index.

… Say what?! 

Stay with us, folks. This post will walk you through why you might want to remove certain webpages from the SERPS (search engine results pages), and exactly how to go about doing it. 

Why You’d Want to Exclude Certain Web Pages From Search Results

There are a number of occasions where you may want to exclude a webpage — or a portion of a webpage — from search engine crawling and indexing.

For marketers, one common reason is to prevent duplicate content (when there is more than one version of a page indexed by the search engines, as in a printer-friendly version of your content) from being indexed.

Another good example? A thank-you page (i.e., the page a visitor lands on after converting on one of your landing pages). This is usually where the visitor gets access to whatever offer that landing page promised, such as a link to an ebook PDF.

Here’s what the thank-you page for our SEO tips ebook looks like, for example:

You want anyone who lands on your thank-you pages to get there because they’ve already filled out a form on a landing page — not because they found your thank-you page in search.

Why not? Because anyone who finds your thank-you page in search can access your lead-generating offers directly — without having to provide you with their information to pass through your lead-capture form. Any marketer who understands the value of landing pages understands how important it is to capture those visitors as leads first, before they can access your offers.

Bottom line: If your thank-you pages are easily discoverable through a simple Google search, you may be leaving valuable leads on the table.

What’s worse, you may even find that some of your highest ranking pages for some of your long-tail keywords might be your thank-you pages — which means you could be inviting hundreds of potential leads to bypass your lead-capture forms. That’s a pretty compelling reason why you’d want to remove some of your web pages from SERPs.

So, how do you go about “de-indexing” certain pages from search engines? Here are two ways to do it.

2 Ways to De-Index a Webpage From Search Engines

Option #1: Add a Robots.txt file to your site.

Use if: You want more control over what you de-index, and you have the necessary technical resources.

One way to remove a page from search engine results is by adding a robots.txt file to your site. The advantage of using this method is that you can get more control over what you are allowing bots to index. The result? You can proactively keep unwanted content out of the search results.

Within a robots.txt file, you can specify whether you’d like to block bots from a single page, a whole directory, or even just a single image or file. There’s also an option to prevent your site from being crawled while still enabling Google AdSense ads to work, if you have them.

That being said, of the two options available to you, this one requires the most technical kung fu. To learn about how to create a robots.txt file, you’ll want to read through this article from Google Webmaster Tools.

HubSpot customers: You can learn how to install a robots.txt file on your website here, and learn how to customize the contents of the Robots.txt file here.

If you don’t need all the control of a robots.txt file and are looking for an easier, less technical solution, then this second option is for you.

Option #2: Add a “noindex” metatag and/or a “nofollow” metatag.

Use if: You want an easier solution to de-indexing an entire webpage, and/or de-indexing the links on an entire webpage.

Using a metatag to prevent a page from appearing in SERPs — and/or the links on a page — is both easy and effective. It requires only a tiny bit of technical know-how — in fact, it’s really just a copy/paste job if you’re using the right content management system.

The tags that let you do these things are called “noindex” and “nofollow.” Before I get into how to add these tags, let’s take a moment to define and distinguish between the two. They are, after all, two completely different directives — and they can be used either on their own, or alongside one another.

What’s a “noindex” tag?

When you add a “noindex” metatag to a webpage, it tells a search engine that even though it can crawl the page, it cannot add the page into its search index.

So any page with the “noindex” directive on it will not go into the search engine’s search index, and can therefore not be shown in search engine results pages.

What’s a “nofollow” tag?

When you add a “nofollow” metatag to a webpage, it disallows search engines from crawling the links on that page. This also means that any ranking authority the page has on SERPs will not be passed on to pages it links to.

So any page with a “nofollow” directive on it will have all its links ignored by Google and other search engines.

When would you use “noindex” and “nofollow” separately vs. together?

Like I said before, you can put add a “noindex” directive either on its own, or together with a “nofollow” directive. You can also add a “nofollow” directive on its own, too.

Add only a “noindex” tag: when you don’t want a search engine to index your web page in search, but you do want it to follow the links on that page — thereby giving ranking authority to the other pages your page links to.

Paid landing pages are a great example of this. You don’t want search engines to index landing pages in search that people are supposed to pay to see, but you might want the pages it links to benefit from its authority.

Add only a “nofollow” tag: when you do want a search engine to index your web page in search, but you don’t want it to follow the links on that page.

There aren’t too many examples of when you’d add a “nofollow” tag to a whole page without also adding a “noindex” tag. When you’re figuring out what to do on a given page, it’s more a question of whether to add your “noindex” tag with or without a “nofollow” tag.

Add both a “noindex” and “nofollow” tag: when you don’t want search engines to index a webpage in search, and you don’t want it to follow the links on that page.

Thank-you pages are a great example of this type of situation. You don’t want search engines to index your thank-you page, nor do you want them to follow the link to your offer and start indexing the content of that offer, either. 

How to Add a “noindex” and/or a “nofollow” metatag

Step 1: Copy one of the following tags.

For “noindex”:

<META NAME=”robots” CONTENT=”noindex”>

For “nofollow”:

<META NAME=”robots” CONTENT=”nofollow”>

For both “noindex” and “nofollow”:

<META NAME=”robots” CONTENT=”noindex,nofollow”>

Step 2: Add the tag to the <head> section of your page’s HTML, a.k.a. the page’s header.

If you’re a HubSpot customer, this is super easy — click here or scroll down for those instructions specific to HubSpot users.

If you’re not a HubSpot customer, then you’ll have to paste this tag into the code on your webpage manually. Don’t worry — it’s pretty simple. Here’s how you do it.

First, open the source code of the web page you’re trying to de-index. Then, paste the full tag into a new line within the <head> section of your page’s HTML, known as the page’s header. The screenshots below will walk you through it.

The <head> tag signifies the beginning of your header: 

Here’s the metatag for both “noindex” and “nofollow” pasted within the header:

And the </head> tag this signifies the end of the header:

Boom! That’s it. This tag tells a search engine to turn around and go away, leaving the page out of any search results.

HubSpot customers: Adding the “noindex” and “nofollow” metatags is even easier. All you have to do is open the HubSpot tool to the page you want to add these tags to, and choose the “Settings” tab.

Next, scroll down to Advanced Options and click “Edit Head HTML.” In the window that appears, paste the appropriate code snippet. In the example below, I’ve added both a “noindex” and a “nofollow” tag since it’s a thank-you page.

Press “Save,” and you’re golden.

Ta Da!

You’ve just magically erased your page from search engine results. Now, you can start capturing more of those lost leads again.

Now, keep in mind that you won’t see results instantaneously. Your changes won’t kick in until the next time a search engine crawls your page. Depending how often you typically publish new pages to your website, it actually could take a few weeks. The more frequently you publish content, the more often search engines will crawl your site. The best way to keep track of how frequently Google visits your website, is to examine your crawl stats in Google Webmaster Tools.

Bottom line: If you notice that your page is still appearing in Google’s search results even with the “noindex” tag, it’s probably because Google hasn’t crawled your site since you added the tag. You can request that Google recrawl your page using the Fetch as Google tool.

Also, note that some search engines’ web crawlers might interpret these directives differently than Google does, so it’s possible that your page might still appear in results from other search engines. But for Google, it’ll work just fine — once it does get around to crawling your website.

Regardless, you’ll be able to sleep a little easier knowing you’ve ultimately made your website a better place for your marketing.

What other tips do you have for de-indexing webpages, and when doing so would be useful for marketers? Share your thoughts in the comments.

 

Marketers considering ads always ask me the same thing: “How much should I spend?”

It’s a good question — an important question — but it’s the wrong question. The right question is: “For every dollar I spend on ads, how much will I get back?”

If you know the answer to that question, or at least have an educated guess, it matters less how much or little you can spend. To help you figure this out, we’re launching a free tool called the HubSpot Advertising ROI Calculator.

This simple calculator allows you to explore how different metrics will impact the profit and ROI you can expect from ads. It will help you get the math right, so whether you have $100 to spend or $100,000, you won’t have to guess at the outcome.  

Ready to give it a try? Click here to check out the HubSpot Advertising ROI Calculator.

How to Plan for Ad Spend Success

Too many marketers tell me about getting burnt by ads. They build an AdWords campaign to rank for important search terms, it drives clicks, traffic, and leads, but ultimately their ad spend outweighs the impact of the ads.

These marketers end up learning a really expensive lesson — one that could be easily prevented by following these simple steps … 

1) Do the math.

Ads aren’t right for everyone. Some industries have extremely high competition with astronomical CPCs. Some products have too low of an average sales price for the economics to work.

To determine if ads are worth your time, run the numbers specific to your business through our calculator. With the tool you can explore how a change in numbers, a lower CPC or higher close rate for example, will impact a campaign. This will give you a better idea of what it will take to create an ads campaign that makes you money.

2) Never set it and forget it.

Try not to look at ads as a shortcut. It’s possible to get to a place with ads where they become a recurring source of profit … but typically that’s not how things start.

Don’t be surprised when a week after your campaign launches your ads aren’t generating tons of profit. Examine the data and make improvements to your ads targeting, creative, and landing pages.

Incremental improvements in clickthrough rates and conversion rates can have a huge impact on profit. Use the ads calculator to explore the impact these changes could have. Watch your ads like a hawk for the first month and you’ll avoid big mistakes.

3) Don’t place your ads in a vacuum.

Ads act like a megaphone to amplify your marketing campaigns and content. The more complete and well developed your campaigns are, the better your ads will perform.

Think about it: Which ads do you think perform better? Those promoting your homepage or those promoting a remarkable piece of content that helps someone?

Bingo. Those promoting awesome content.

Campaigns that use ads should be treated like other campaigns. Establish your goals first, build great content, focus your message and optimize your landing pages, then figure out how ads can help amplify your message. Taking this combined approach and only using ads as a strategic component of your marketing campaigns will pay off.

So how much should you actually spend on ads?

Assuming you’ve run the numbers and now know what you can expect in terms of profit and ROI, it’s time to launch a campaign. To do this you’ll have to make two budget decisions, regardless of where you are advertising:

• Total campaign budget or duration: How much do you plan to spend in total? How long will the ads run for? Be aware that if you don’t set a limit there will be no maximum or end date.
• Daily budget: How much do you want to spend a day?

There are two ways I see most marketers use ads, short-term and long-term. Let’s take a look a both …

Short-term ads. Budget = at least a few hundred dollars over a month.

When marketers use ads for a short-term goal, it’s usually to jump start a campaign or boost content that needs a bump. These ad spends are generally smaller and shorter, but can be large.

If you have a few hundred dollars to spend, spend it this way. Create a social post that promotes a piece of content and then use your ads to boost the post. Facebook, Twitter, and LinkedIn all have good solutions for this. Make sure you use the social network that gives you the best targeting for your persona and returns the most valuable leads. Measure this by assessing the quality of the leads generated after the campaign.

Long-term ads. Budget = at least a few thousand dollars over a quarter.

Ads can be a great solution in a pinch but if you really want to use ads strategically build them into your overall marketing strategy. This means more consistent, quarterly ad spends.

Consider how potential customers make purchase decisions and use ads to influence them. This may mean always relying on search ads or retargeting to make sure prospects find you when they are ready to buy. You’re more likely to accumulate better data taking this approach, which will allow you to get more sophisticated in how you optimize your ad spend.

Whatever budget you choose and whichever approach you take, make sure to ask a lot of questions about your ad spend. Even with the right approach, it’s easy to burn through money fast. Safeguard yourself by knowing what to expect and having clear expectations.

You started your own agency because you wanted a different type of lifestyle — you wanted to build something of your own. You were sick of the lack of creativity in the work at your company, and the singular focus on margins and profits were draining your love for the industry. 

But running your own agency — whether it’s just you or you’ve hired a few or more people — is a completely different challenge. Now, you’re responsible for everything, from the work to the happiness of clients to hiring to the state of your bill with the electric company. 

And you’ve run into a few too many troublesome situations in your adventure in building an agency — ones you wish you could have avoided.

To prevent making the same mistakes as many others new to ownership, check out these common problems. 

5 Common Mistakes New Agency Owners Make 

1) Not getting a contract signed before starting work.

Many new agency owners are working on projects from friends or friends of friends. Others are simply too afraid of losing the deal to ask a prospect to sign on the dotted line. And some simply get comfortable after the third or fourth project, and they think a legal document is no longer necessary: The client has paid up before with no problems. What could happen to make that change?

They forgo this step until one day, they get completely burned by a client who won’t answer their calls and definitely won’t pay up.

The key is making the contract a seamless part of the sales process. Whether you are a one-person agency or a 25-person shop, the contract should be an essential step prior to starting any work. It confirms that the client is committed to the project, and it clarifies payment terms and schedules. It should also outline the scope so that it is clear what’s included in the project and what’s not.

It protects both you and your client, and it will help you avoid clients who might take advantage of your status as an inexperienced business owner.

2) Discounting projects from the start.

Many agency owners just starting out are hungry for new business and for clients with whom they can do amazing work.

And so before the client even asks for a discount or says that the price is outside of their budget, they start chipping away at their bid. Or in an effort to make sure they win the project, they slash their rates and come in way under the normal price.

By starting out the relationship and saying “My work is not worth the price I set,” you’re losing credibility with the client from the start. It can also be seen as desperate by a client who knows you’ll do whatever it takes to work with them.

3) Letting emotions get in the way.

You left your day job and convinced your family to trust in you as you pursue your desire to run your own agency. This is your future. And everything is personal to you.

But to most of your clients, it’s not personal. And you need to remember this fact when you become frustrated about a missed client deadline that pushes back the launch — and your payment — by another month or when you feel hurt by the client’s lack of response to your carefully crafted proposal.

It’s far too easy for new agency owners to want to vent these feelings in an email or during a call when business interactions don’t go the way they think they should. However, this is a business where your reputation matters. Burning bridges won’t help you lay a strong foundation for future growth.

4) Not setting realistic timelines.

When trying to sell a client on a project, it’s easy to want to over-promise: “Sure, we can do that website redesign in three weeks. No problem at all.” Unless you and your team skip eating, sleeping, and doing anything else in that timeframe, it’s unlikely you can meet that deadline. But the client wants it in three weeks. You need to do whatever takes to deliver. You’re in the client service business, right?

Wrong. This attitude will only result in you delivering either shoddy work or missing deadlines — two serious problems that will put the relationship at risk.

Get ahead of this curve by telling clients up front the minimum amount of time required to turn around specific projects. Stick to these timelines. If you deliver more quickly, then you’ll have an even happier client. If you can’t meet the deadline with your current workload, maybe it’s time to hire another employee or work with a freelancer, or maybe you just need to say “no” to more client work.

Don’t overwhelm your team by taking on too much and over-promising on the time it takes to create quality, results-oriented work.

5) Never saying “no” to client work. 

You want to get your agency’s name out there, and that means it can be difficult to say to a prospect, “I don’t do that,” or “That’s not my area of expertise.”

So you just say “yes” instead and hope that you can find a freelancer who can do the work within the budget you proposed. While outsourcing work is a normal practice, you need to be able to vouch for the freelancer’s quality of work — at the end of the day, it’s ultimately your responsibility. That’s a risky way to begin any new relationship and make a name for your agency.