MOSS MVP

I've moved my blog to http://blog.falchionconsulting.com!. Please update your links. This blog is no longer in use--you can find all posts and comments at my new blog; I will no longer be posting to this site and comments have been disabled.

Thursday, January 15, 2009

Recalculating Usage Statistics via STSADM or PowerShell

I was perusing through the SharePoint forums the other day and I came across an issue that someone was having with the usage statistics information for their My Sites site collections.  When they viewed the usage data (~site/_layouts/usage.aspx) they were seeing incorrect information.  I’m not really sure why the numbers were wrong but fixing them turned out to be pretty easy.  There’s a public method called “RecalculateStorageUsed” that, when called, will recalculate the usage statistics for the site collection.  I decided to do some digging within the SharePoint code and what I found was rather interesting – Microsoft created an stsadm command that would allow you to call this method via stsadm for a site collection – but they didn’t publish the command via any config file so even though the code is there, you can’t use it.  I tried to do some more poking around to see if there was a timer job or something that either called this method or one of the internal SPRequest methods that actually does the work but unfortunately I didn’t find anything that would help identify why the numbers weren’t correct.

So, knowing that Microsoft had started the creation of such a command but didn’t finish I decided that I’d go ahead and “finish” it for them – but better, naturally :).  You can now use my gl-recalculateusage command and pass in various scopes (Farm, WebApplication, or Site) and it will call the aforementioned method for each site collection within the specified scope.

The complete code for the command is included below:

   1: using System;
   2: using System.Text;
   3: using Lapointe.SharePoint.STSADM.Commands.OperationHelpers;
   4: using Lapointe.SharePoint.STSADM.Commands.SPValidators;
   5: using System.Collections.Specialized;
   6: using Microsoft.SharePoint;
   7: using Microsoft.SharePoint.Administration;
   8:  
   9: namespace Lapointe.SharePoint.STSADM.Commands.SiteCollectionSettings
  10: {
  11:     /// <summary>
  12:     /// 
  13:     /// </summary>
  14:     public class RecalculateUsage : SPOperation
  15:     {
  16:         /// <summary>
  17:         /// Initializes a new instance of the <see cref="RecalculateUsage"/> class.
  18:         /// </summary>
  19:         public RecalculateUsage()
  20:         {
  21:             SPParamCollection parameters = new SPParamCollection();
  22:             parameters.Add(new SPParam("url", "url", false, null, new SPUrlValidator()));
  23:             parameters.Add(new SPParam("scope", "s", false, "site", new SPRegexValidator("(?i:^Farm$|^WebApplication$|^Site$)")));
  24:  
  25:  
  26:             StringBuilder sb = new StringBuilder();
  27:             sb.Append("\r\n\r\nRecalculates usage statistics for the given site(s).\r\n\r\nParameters:");
  28:             sb.Append("\r\n\t[-scope <Farm | WebApplication | Site>]");
  29:             sb.Append("\r\n\t[-url <url>]");
  30:  
  31:             Init(parameters, sb.ToString());
  32:         }
  33:  
  34:         /// <summary>
  35:         /// Gets the help message.
  36:         /// </summary>
  37:         /// <param name="command">The command.</param>
  38:         /// <returns></returns>
  39:         public override string GetHelpMessage(string command)
  40:         {
  41:             return HelpMessage;
  42:         }
  43:  
  44:         /// <summary>
  45:         /// Executes the specified command.
  46:         /// </summary>
  47:         /// <param name="command">The command.</param>
  48:         /// <param name="keyValues">The key values.</param>
  49:         /// <param name="output">The output.</param>
  50:         /// <returns></returns>
  51:         public override int Execute(string command, StringDictionary keyValues, out string output)
  52:         {
  53:             output = string.Empty;
  54:             Verbose = true;
  55:  
  56:             string scope = Params["scope"].Value.ToLowerInvariant();
  57:  
  58:             SPEnumerator enumerator;
  59:             if (scope == "farm")
  60:             {
  61:                 enumerator = new SPEnumerator(SPFarm.Local);
  62:             }
  63:             else if (scope == "webapplication")
  64:             {
  65:                 enumerator = new SPEnumerator(SPWebApplication.Lookup(new Uri(Params["url"].Value.TrimEnd('/'))));
  66:             }
  67:             else
  68:             {
  69:                 // scope == "site"
  70:                 using (SPSite site = new SPSite(Params["url"].Value.TrimEnd('/')))
  71:                 {
  72:                     Recalculate(site);
  73:                 }
  74:                 return OUTPUT_SUCCESS;
  75:             }
  76:  
  77:             enumerator.SPSiteEnumerated += new SPEnumerator.SPSiteEnumeratedEventHandler(enumerator_SPSiteEnumerated);
  78:             enumerator.Enumerate();
  79:  
  80:             return OUTPUT_SUCCESS;
  81:         }
  82:  
  83:         /// <summary>
  84:         /// Validates the specified key values.
  85:         /// </summary>
  86:         /// <param name="keyValues">The key values.</param>
  87:         public override void Validate(StringDictionary keyValues)
  88:         {
  89:             if (Params["scope"].Validate())
  90:             {
  91:                 Params["url"].IsRequired = true;
  92:                 Params["url"].Enabled = true;
  93:                 if (Params["scope"].Value.ToLowerInvariant() == "farm")
  94:                 {
  95:                     Params["url"].IsRequired = false;
  96:                     Params["url"].Enabled = false;
  97:                 }
  98:  
  99:             }
 100:             base.Validate(keyValues);
 101:         }
 102:  
 103:         /// <summary>
 104:         /// Handles the SPSiteEnumerated event of the enumerator control.
 105:         /// </summary>
 106:         /// <param name="sender">The source of the event.</param>
 107:         /// <param name="e">The <see cref="Lapointe.SharePoint.STSADM.Commands.OperationHelpers.SPEnumerator.SPSiteEventArgs"/> instance containing the event data.</param>
 108:         private void enumerator_SPSiteEnumerated(object sender, SPEnumerator.SPSiteEventArgs e)
 109:         {
 110:             Recalculate(e.Site);
 111:         }
 112:  
 113:         /// <summary>
 114:         /// Recalculates the specified site.
 115:         /// </summary>
 116:         /// <param name="site">The site.</param>
 117:         private void Recalculate(SPSite site)
 118:         {
 119:             Log("Recalculating {0}", site.Url);
 120:             site.RecalculateStorageUsed();
 121:             using (SPSite site2 = new SPSite(site.ID))
 122:                 Log("Storage updated from {0} to {1} (in bytes)\r\n", site.Usage.Storage.ToString(), site2.Usage.Storage.ToString());
 123:         }
 124:     }
 125: }

The help for the command is shown below:

C:\>stsadm -help gl-recalculateusage

stsadm -o gl-recalculateusage


Recalculates usage statistics for the given site(s).

Parameters:
        [-scope <Farm | WebApplication | Site>]
        [-url <url>]

The following table summarizes the command and its various parameters:

Command Name Availability Build Date
gl-recalculateusage WSS v3, MOSS 2007 Released: 1/15/2009

Parameter Name Short Form Required Description Example Usage
url   Yes if scope is not Farm URL of the web application or site collection. -url http://portal
scope s No – defaults to site The scope to use.  Valid values are “Farm”, “WebApplication”, and “Site” -scope site

-s site

The following is an example of how to recalculate the usage statistics for all the site collections within the farm:

stsadm -o gl-recalculateusage -scope farm

The following is an example of the output you might see after running the above command:

C:\>stsadm -o gl-recalculateusage -scope farm

Recalculating http://mysites
Usage updated from 425744 to 425744

Recalculating http://mysites/personal/spadmin
Usage updated from 588790 to 588790

Recalculating http://portal
Usage updated from 3249962 to 3249962

Recalculating http://sspadmin/ssp/admin
Usage updated from 642686 to 642686

Recalculating http://sharepoint1:1234
Usage updated from 18244961 to 18284043

Operation completed successfully.

If you wanted to do this exact same thing using PowerShell you could do the following:

PS W:\> foreach ($site in Get-SPSite -url *) {
>> $origStorage = $site.SPBase.Usage.Storage
>> $site.SPBase.RecalculateStorageUsed()
>> $site.SPBase.Dispose()
>> $tempSite = $site.GetSPObject()
>> $newStorage = $tempSite.Usage.Storage
>> Write-Host $tempSite.Url Updated from $origStorage to $newStorage
>> $tempSite.Dispose()
>> }
>>
http://mysites Updated from 425744 to 425744
http://mysites/personal/spadmin Updated from 588790 to 588790
http://portal Updated from 3249962 to 3249962
http://sspadmin/ssp/admin Updated from 642686 to 642686
http://sharepoint1:1234 Updated from 18284043 to 18284043
PS W:\>

Note that I used the SPBase property initially to get the current storage and then to call the RecalculateStorageUsed method.  You then must dispose of this object.  I then used the GetSPObject() method to get a new copy of the SPSite object because I can’t use the original copy as the usage data would be cached.  I can now use this new object to get the new usage data which I then write to the host and then I’m free to dispose of the object.  Also, because the SPSiteInfo objects that are returned by the Get-SPSite cmdlet do not require disposal you can easily pass the filter the results of that command before looping through the returned collection without worrying about disposing of items.

6 comments:

Anonymous said...

Any chance you can put a compiled version on there for download? Or put it onto codeplex as a project?

Gary Lapointe said...

The source code and the WSP can be downloaded from any page within my blog (see the links at the top of the page). If you don't need the source just download the WSP and deploy to your farm.

Anonymous said...

Gary,
Thanks for making your tools available. I was thinking that this 're-calc' was how I could make MOSS 2007 re-look at all of the logs and re-calc or re-create the daily/monthly site collection usage stats. Last month we had a crash of search on one of our SSPs and created a new SSP and shifted the sites across...all fine - search working, etc. BUT when we came to check the previous month's stats they (and current stats) show almost NO hits although many folks use the box and the actual 'usage logs' are about the same size as before (heaps of data). If this command is not the correct one for THOSE usage stats - would you know how that is done? I have searched through the web/technet and poked around the diretories and interface but cannot see any way to make MOSS 'look' at the older stats :(. Cheers, Ken Thomason, QUT, Brisbane, Australia (org. Florida)

Gary Lapointe said...

Honestly I think you're going to be out of luck on this - I don't really have time to dig into it but I'm pretty sure that the stats associated with one site cannot be "migrated" to another.

Anonymous said...

Hi Gary,

I got this message:
Recalculating http://site.com
The user does not exist or is not unique.

Do you have any idea why I am getting this message?

Thanks for your help!

Gary Lapointe said...

That's a new one for me but my guess would be that it's permissions related - are you using a farm admin account?