#1 WordPress forum plugin created by gVectors Team

wpForo – WordPress Forum Plugin
  • Home
  • Forum
  • Migrate to wpForo
  • Addons
  • Addons Demo
  • Documentation

Forum

Home | Forum

wpDiscuz - WordPress Comment Plugin
  • Forums
  • Members
  • Recent Posts
Forums
Main Support Forums
How-to and Troubles...
Robots TXT and Goog...
 
Share:
Share
Tweet
Share
Notifications
Clear all

Robots TXT and Google Search Console

    Last Post
RSS

Macky
Posts: 777
 Macky
Beta Tester
Topic starter
March 16, 2021 11:28 pm
(@percysgrowroom)
Prominent Member
Joined: 3 years ago

I need to reduce the amount of pages google crawls when it comes to my site. 

I would like to restrict the crawl to the sections with most information, to make sure google crawl the best stuff. 

I can do this in the tool section of the forum, or through robots txt, but would this show as an error on my google search console? Or would it not show at all? 

Also, does anyone have an example of what robots txt they use? Any advice on this would be massively appreciated.

Check out my Wpforo forum, lots of customisations and add ons 🙂
Percys Grow Room

6 Replies
alexas
Posts: 34
 alexas
March 17, 2021 9:56 am
(@alexas)
Eminent Member
Joined: 5 years ago

I have a standard robot, I tried everything and there was even a robot 25 lines each. The load is minimal. Average attendance of 400 people per day. You should have caching enabled in the forum settings and no plugins are needed except for kaptha bank. I hope I didn't hurt you!

User-agent: *
Disallow: /wp-admin/
Disallow: /r/
Disallow: /wp-json/

https://manval.ru/

Reply
Macky liked
2 Replies
dimalifragis
 dimalifragis
Beta Tester
(@dimalifragis)
Joined: 2 years ago

Noble Member
Posts: 1115
March 17, 2021 12:29 pm
Reply toalexasalexas
Posted by: @alexas

I have a standard robot, I tried everything and there was even a robot 25 lines each. The load is minimal. Average attendance of 400 people per day. You should have caching enabled in the forum settings and no plugins are needed except for kaptha bank. I hope I didn't hurt you!

User-agent: *
Disallow: /wp-admin/
Disallow: /r/
Disallow: /wp-json/

wp-json is already excluded i think.

$this->send_header( 'X-Robots-Tag', 'noindex' );

 Also this should be WP robots.txt

Allow: /wp-admin/admin-ajax.php

Reply
Macky liked
Macky
 Macky
Beta Tester
(@percysgrowroom)
Joined: 3 years ago

Prominent Member
Posts: 777
March 17, 2021 1:04 pm
Reply todimalifragisdimalifragis
alexas

@dimalifragis Thanks bro, ill add that 🙂 

 

Check out my Wpforo forum, lots of customisations and add ons 🙂
Percys Grow Room

Reply
Macky
Posts: 777
 Macky
Beta Tester
Topic starter
March 17, 2021 1:06 pm
(@percysgrowroom)
Prominent Member
Joined: 3 years ago

This is how mine look:

User-Agent: *
Allow: /wp-admin/admin-ajax.php
Allow: /wp-content/uploads/
Disallow: /wp-content/plugins/
Disallow: /wp-admin/
Disallow: /readme.html
Disallow: /refer/
Disallow: /forum/profile/

Sitemap: https://percysgrowroom.com/forum/sitemap.xml
Sitemap: https://percysgrowroom.com/sitemap_index.xml

I added the profile bit last night, and just added that line told me too, is this ok do you think?

Thanks for the input guys, I really appreciate the help 

Check out my Wpforo forum, lots of customisations and add ons 🙂
Percys Grow Room

Reply
alexas and dimalifragis liked
dimalifragis
Posts: 1115
 dimalifragis
Beta Tester
March 17, 2021 1:11 pm
(@dimalifragis)
Noble Member
Joined: 2 years ago

@percysgrowroom Seems right, still i'm no expert in all that.

Reply
Macky liked
1 Reply
Macky
 Macky
Beta Tester
(@percysgrowroom)
Joined: 3 years ago

Prominent Member
Posts: 777
March 17, 2021 3:00 pm
Reply todimalifragisdimalifragis

@dimalifragis More of an expert than me mate lol, thanks for your help

Check out my Wpforo forum, lots of customisations and add ons 🙂
Percys Grow Room

Reply
  All forum topics
  Previous Topic
Next Topic  

Forum Search

Join Us!

Download wpForo plugin
on WordPress.org

wpForo Addons

wpforo-private-messages wpforo-advanced-attachments-128x128 wpforo-embeds-128x128 wpForo User Custom Fields addon wpForo – Blog Cross Posting addon wpForo Ads Manager wpForo – WooCommerce Memberships Integration wpForo Emoticons wpForo – Tenor GIFs Integration
View all Addons »

Recent Topics

  • Font Awesome Icons

    By seancallahan, 11 hours ago

  • Notification bell order

    By teepeenomad, 2 days ago

  • Not İndexing

    By selinahub, 2 days ago

  • H1 to H6 title tags : remove

    By venus, 3 days ago

  • Can't access forum dashboard 504 gateway time-out

    By Joselito, 3 days ago

  • Sharing with Facebook and Twitter is kind of broken

    By seancallahan, 3 days ago

Topic Tags

  • css52
  • translation50
  • plugin conflict48
  • seo47
  • new features45
  • buddypress43
  • avatar41
  • moderation40
  • ultimate member40
  • login40
  • forum40
  • registration37
  • threaded layout35
  • cache34
  • editor33
  • menu33
  • shortcode32
  • spam31
  • forum accesses30
  • widget29
View all tags (2188)

Recent Posts

  • Font Awesome Icons

    To start, I do not have any caching setup on the site o...

    By seancallahan, 11 hours ago

  • RE: Is there a way to add safety features?

    @chris Is there a way for me to install 2.0 on my site ...

    By seancallahan, 14 hours ago

  • RE:

    By Anonymous, 53 years ago

  • RE: Notification bell order

    @tutrix Brilliant, thank you for this! Although, it ...

    By teepeenomad, 17 hours ago

  • RE: wpForo 2.0.0.1 beta version is released for beta testers!

    Added, thank you.

    By Robert, 2 days ago

  • RE: Post title length error.

    Hi @hasanaytac, Please provide your forum URL.

    By Chris, 2 days ago

  • RE: Health.gd Health website

    Nice forum, still you have an issue / crash: Uncaught...

    By dimalifragis, 2 days ago

Share:
Share
Tweet
Share
  Forum Statistics
21 Forums
9,925 Topics
50.2 K Posts
8 Online
47.6 K Members

Latest Post: Font Awesome Icons Our newest member: Смотреть Мультфильм Бэймакс 1 ... Recent Posts Unread Posts Tags

Forum Icons: Forum contains no unread posts Forum contains unread posts

Topic Icons: Not Replied Replied Active Hot Sticky Unapproved Solved Private Closed

Powered by wpForo | Copyright © 2016-2022 gVectors Team
Copyright Registration Service - Click here for more information or to register work
wpForo is Registered with the IP Rights Office
Copyright Registration Service

Ref: 4477265538
  • Home
  • Forum
  • Migrate to wpForo
  • Addons
  • Addons Demo
  • Documentation