Login
0

Core URL Rewrites Optimization

v1.9.3
Core URL Rewrites Optimization
$54.00
Pro installation
  • 1146
  • 256
  • 5

It’s a matter of common knowledge that with time the Magento 'core_url_rewrite' table continuously increases in size, which in turn affects website’s performance and reduces processing efficiency of data bases. We’ve figure out the source of the problem and developed a sublime solution for you.

Description

The reason for increasing size of the foregoing table is duplicating/intersecting URL Keys of products and categories. To be more specific, default Magento renames duplicated URLs into 'product-1.html', 'product-2.html', and so on, by adding a respective sequence number.

Surprisingly, whenever admin user tries to save a product or run 'Catalog URL Rewrites' reindex, all URLs generated before get suffixed again without any reasonable cause:
product-1.html -> product-3.html -> product-5.html
product-2.html -> product-4.html -> product-6.html

Additionally, for the previous URL to work, Magento’s code adds several redirect records:
product-1.html -> product-5.html
product-3.html -> product-5.html
product-2.html -> product-6.html
product-4.html -> product-6.html

As if that wasn't enough, in case of multi-category products, a number of the records is multiplied by a number of the related categories. How about geometric series here?

With every reindex or product/category edit, this happens infinitely. Just think of it for a moment: the table gets continuously pumped, a reindex operation takes forever to finish, and records increase by almost half.


How it works

1) To start with, admin user is to get rid of excess redirects
• System > Configuration > SafeMage: URL Rewrites Optimization > Clearing Settings of 'core_url_rewrite' Table
In most cases, we'd recommend choosing the 'Remove All Excess Redirects' option under Clearing Mode. You should not be anxious about Google or your customers not finding a previously indexed product thanks to the setting:
• System > Configuration > SafeMage: URL Rewrites Optimization > Routing of Pages Ending with a Digit: Enable Routing = Yes
What this means is, if an old URL like [product_url_key]-297823 or [category_url_key]-1126 is recognized by your Magento as 404, the extension then tries to find such an URL without digits at its end, and, if succeeded, enables 301 redirect for it:
product-297823.html > product.html
category-1126.html > category.html

2) Next, it's time to reindex data
• System > Index Management > Catalog URL Rewrites > Reindex Data
URLs of 'Disabled' or 'Not Visible Individually' products, inactive categories, or disabled store views will not be added to 'core_url_rewrite'. This will enable to decrease the table size even more significantly, especially if the store includes many Configurable or Bundle products. Note that if the final table includes too many entries you'd better run reindex via SSH:
php shell/indexer.php --reindex catalog_url

3) There is something to be accentuated: the extension is not a one-time pill for your system. It automatically fixes several Magento bugs during Catalog URL Rewrites reindex or while saving product/category. So the table won’t be increasing in size without real necessity.

4) A handy 'Unique URL Key Verification' setting can be enabled to prevent admin user to enter duplicated URL keys.

5) To get even more out of the optimization, you can exclude categories in product URLs. After disabling the setting below, the Catalog URL Rewrites must be reindexed.
• System > Configuration > CATALOG: Catalog > Search Engine Optimizations > Use Categories Path for Product URLs

Real examples? Once upon a time, we managed to lower the number of table records from 27 million to mere 45 thousand (this is terrific 600 times less). The table has remained on the similar size level until now.


Experimental Feature

Magento is also known to keep excessive amount of data in DB that is related to each store view. The same applies to URLs in the 'core_url_rewrite' table. This is why we built an experimental, yet well-tested feature to rectify the situation:
• System > Configuration > SafeMage: URL Rewrites Optimization > Make URLs for Categories and Products Global: Enable Global URL Keys

It is capable of reducing the size of the "core_url_rewrite" table N times (where N is a number of active store views).

Be aware this is an experimental feature suitable not for every store. Its core is ability to transform URL Keys for categories and products to Global ones, i.e. URL Path of products and categories for all store views would become identical (note this doesn't apply to store code in URL).

Existing records with store_id > 0 get removed and new records with store_id = 0 get added from/to the "core_url_rewrite" table respectively. Note, if you run a one store view shop, the setting won't work for you.

Features

  • Optimization procedure guarantees minimum possible size of 'core_url_rewrite' (up to 10000 times less records)
  • Fixes various Magento bugs that surface during Catalog URL Rewrites reindex or when saving product / category
  • Fixes an issue with reindex when categories are omitted in product URLs:
    System > Configuration > CATALOG: Catalog > Search Engine Optimizations > Use Categories Path for Product URLs = NO
  • Keeps the 'core_url_rewrite' table on the same size level
  • URLs of 'Disabled' or 'Not Visible Individually' products / inactive categories / disabled store views will not end up in 'core_url_rewrite'
  • Safe clearing of the table enables removal of excess redirects (with / without set qty of latest redirects to keep or keeping visited URLs from "log_url_info" table)
  • Enables unique URL Key verification
  • Restores URLs ending with a digit
  • Includes schedule clearing
  • Checks and clears log tables
Please share your results of the 'core_url_rewrite' table optimization using our product, i.e. records: before > now.

Screenshots

System > Configuration > SAFEMAGE > Extensions & Contact System > Configuration > SAFEMAGE > URL Rewrites Optimization System > Configuration > SAFEMAGE > URL Rewrites Optimization System > Configuration > SAFEMAGE > URL Rewrites Optimization Catalog > Manage Products > Your Product

Compatibility

Magento CE 1.4.x, 1.5.x, 1.6.x, 1.7.x, 1.8.x, 1.9.x
Magento EE 1.9.x, 1.10.x, 1.11.x, 1.12.x, 1.13.x, 1.14.x

Support

We provide comprehensive product support for all the Magento extensions available on this site starting from the moment of your purchasing them. If you need help on the extensions installation or on any issues relating to our products, our dedicated team of proven technical experts is at your service, ready to assist you at any time.

Changelog

new feature bugfix
v.1.9.3 (Oct 29, 2019)
Fixed the 'restore URLs' process.
v.1.9.2 (Jul 14, 2019)
Improved support for URLs without html suffix.
v.1.9.1 (Jul 10, 2018)
Addressed an issue with 'Use Categories Path for Product URLs' set to 'NO';
Limited 'Unique URL Key Verification' to products only.
v.1.9.0 (Jun 21, 2018)
Enable Routing of Products with Category Path in URL.
v.1.8.1 (Jan 31, 2018)
Addressed a cron-related issue.
v.1.8.0 (Nov 1, 2017)
Added a new 'Routing of Pages Ending with a Digit' setting.
v.1.7.4 (Sep 25, 2017)
Improved optimization results when the ‘Use Categories Path for Product URLs’ setting is set to ‘No’.
v.1.7.3 (Aug 14, 2017)
Added compatibility with Mageworx SEO Suite.
v.1.7.2 (Jul 14, 2017)
Addressed an issue with rewrites for disabled categories.
v.1.7.1 (May 13, 2017)
Improved reindex process.
v.1.7.0 (Jan 27, 2017)
Added Global URL feature (experimental).
v.1.6.0 (Jan 24, 2017)
Fixed a Magento issue with reindex when the following setting is set to No: System > Configuration > CATALOG > Catalog > Search Engine Optimizations > Use Categories Path for Product URLs;
Addressed an issue with categories reindex;
Addressed minor compatibility issue with Magento CE 1.6.x.
v.1.5.0 (Jan 16, 2017)
Added ability to omit disabled stores.
v.1.4.0 (Dec 16, 2016)
Added Restore and Reindex buttons.
v.1.3.0 (Dec 10, 2016)
Added additional information about Log Tables;
Code improvements.
v.1.2.0 (Dec 7, 2016)
Added automatic clearing by Cron.
v.1.1.0 (Dec 6, 2016)
Added new clearing mode 'Remove only with Digit at End'.
v.1.0.0 (Sep 29, 2016)
Initial release.

Reviews(6)

    Looks very promising
    We are trying your module and it looks very promising.
    We have a lots of store views (98) and a huge catalog so our core_url_rewrite table has 7.8 million records weighing about 8GB (data + index). Using your module we get ~48000 records. We are testing very extensively to make sure no features are lost.
    Besides trimming our database a lot it also saved us many hours of development and research to get to this solution.
    by Alex April 11, 2020
    Reduced our URL rewrites index by 98%!
    I have to recommend this extension and the team behind it. It reduced the total number in URL rewrites table from 394313 to just 7688 entries. That's a reduction by 98%! It also drastically improved the time needed for reindex. Before reindexing always caused our frontend to not responding. Which is a bad thing! This solved it for us and the support is really excellent. Thank you! :)
    by Sebastian August 19, 2019
    Get this extension
    Get this extension or you will suffer from some weird URL rewrite bugs Magento has had for quite some time. We found out by clicking the product links in some of our social posts that lead to 404 not found pages, especially those with duplicate URL keys. Lost so many potential social buyers from Pinterest and Facebook. This is a no-brainer for all Magneto stores, get it by all means!
    Princessly
    by Yang August 24, 2018
    The bomb
    Your extension rewrites optimization is the bomb, fixes a very big problem in magento, thanks alot!
    Five Stars!
    by Jason Keys January 18, 2018
    Very good extension
    This is a very good extension, with a little help the extension reduced the amount of rewrite url's from 2 milion to 80K!
    With nice 301 rewrites from digit urls to non-digit urls.
    Explended service!
    by Bartvriend November 5, 2017
    Cleared about 15 million records!
    I got 22 thousand products in the store, mostly configurable (3500) and simple (18500, not visible individually) ones. In several years, the table with URLs has grown greatly and began affecting page load time. Once I installed the extension and ran it, the table shrank down to 5 thousand records. Unbelievable!
    The URL’s of the products not visible individually are not being added to the "core_url_rewrite" table anymore. In fact, it’s an odd thing to add such URL’s to the table.
    I also played around the settings Visibility and Status to make sure a record is added when a product is Enabled and Visible, and the former is removed when the latter isn’t enabled.
    The module works like charm, even without additional re-index. Respect!
    by moodfabrics.com November 16, 2016
Write Your Own Review

Only registered users can write reviews. Please, log in or register