...
Using the OSGi dependency mechanism in GX WebManager, this new UrlProvider service is automatically picked up by the SearchService in the wmasolrsearch add-on, and its list of url's is appended to the default list of GX WebManager.
Changing or excluding url's
If you want to change the url's determined by the default UrlProviderof GX WebManager, you can do this by first capturing these url's, changing them, and then feeding them to the SearchService in the wmasolrsearch add-on:
| Code Block | ||||
|---|---|---|---|---|
| ||||
private UrlProvider m_urlProvider; // Injected by OSGi
private SearchService m_searchService; // Injected by OSGi
private void index(boolean fullIndex, boolean clearRest) {
// Get all the url's from GX WebManager that should be indexed
String[] urls = m_urlProvider.getUrls(indexFullContent);
// Update the url's in the list as you see fit
...
// Index the urls allowing all hostnames and setting the follow-url depth to 0.
// The SearchService class also has a different indexPages method in which you can specify
// the allowed hostnames and depth.
m_searchService.indexPages(urls, allowedHostNames.split(","), 0, m_configUtil.getClearRest());
}
|
When you do this, you normally do not want the default indexer task of GX WebManager to run. You can disable it by emptying the wmasolrsearch.crontabschedule configuration setting on the GX WebManager setup page.
| Code Block | ||||
|---|---|---|---|---|
| ||||
// Get all the url's that should be indexed
String[] urls = m_urlProvider.getUrls(indexFullContent);
LOG.info("The url provider returned " + urls.length + " url's to index.");
// Add, remove or change the url's
urls = updateUrls(urls);
LOG.info("Updating the url's resulted in " + urls.length + " url's to index.");
// Index the urls
String allowedHostNames = m_configUtil.getAllowedHostNames();
if (allowedHostNames != null && allowedHostNames.trim().length() > 0) {
m_searchService.indexPages(urls, allowedHostNames.split(","), 0, m_configUtil.getClearRest());
} else {
// Allow all hostnames and set the depth to 0
m_searchService.indexPages(urls, m_configUtil.getClearRest());
} |
See also the attached example: urlProviderService.zip