MOSS MVP

I've moved my blog to http://blog.falchionconsulting.com!. Please update your links. This blog is no longer in use--you can find all posts and comments at my new blog; I will no longer be posting to this site and comments have been disabled.

Thursday, November 1, 2007

Import, Export, Copy, and Delete List Items

As part of my upgrade I've discovered a need to be able to add items to various lists. I considered a couple of approaches including creating a command that would allow me to specify the field names and values that I wanted to add to the list. Eventually though I concluded that the best and most flexible approach was to use the deployment API to export list items and then import them. The nice part about this approach was that the code was very similar to what I did for exporting and importing of lists. Another benefit is that if you are importing list items into a list that does not have all the fields needed those fields will be automatically created without any extra coding.

I originally set out to just create the export and import commands and then decided to create a delete command so that I could clean up all my test entries. Once I had the export and import done creating a copy command was really simple as it just wraps up those commands. The code to work with the content deployment API is pretty straightforward. Stefan Goßner does a really good job at explaining how to use the API in his blog Deep Dive into SharePoint Content Deployment and Migration API so I won't reiterate it here. The commands I created are detailed below.

1. gl-exportlistitem

This command works almost identically to the gl-exportlist command that I created earlier. The main difference is that it takes in an optional "id" parameter which can be a single list item ID or a comma separated list of list item IDs. The syntax of the command can be seen below:

C:\>stsadm -help gl-exportlistitem

stsadm -o gl-exportlistitem

Exports a list item or items.

Parameters:
        -url <list view url>
        -filename <export file name>
        [-id <list item ID (separate multiple items with a comma)>]
        [-overwrite]
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-versions <1-4>
                1 - Last major version for files and list items
                2 - The current version, either the last major or the last minor
                3 - Last major and last minor version for files and list items
                4 - All versions for files and list items (default)]
        [-cabsize <integer from 1-1024 megabytes> (default: 25)]
        [-nofilecompression]
        [-quiet]

Here's an example of how to export 3 items from the site directory list:

stsadm –o gl-exportlistitem -url "http://intranet/sitedirectory/siteslist/AllItems.aspx" -id "1,2,3" -filename "c:\SiteDirItems" -includeusersecurity -versions 4 -nofilecompression

Running the above will create a folder called "SiteDirItems" in the root c: drive. You'll also find a log file there containing the same information that was dumped to the console. This same output can then be used by the gl-importlistitem command to copy the list items to another list in any site collection (note that the target could be on any web app in any farm). To export all items in a list simply omit the "id" parameter.

2. gl-importlistitem

This command has almost exactly the same parameters as the built in "import" command. One additional parameter of note is the "-retargetlinks" parameter. If this parameter is specified then you must also specify the "-sourceurl" parameter which corresponds to a view of the source list. The "retargetlinks" parameter tells the code to find any links that point to items in the source list and make them point to the target list.

This becomes helpful in a move operation. Note that only items identified by the list items BackwardLinks collection will get modified (unfortunately I'm not 100% clear about when and how this collection is set so I don't claim that using this will fix every link to your items, though I do have another command that will loop through every field of every item of every list of every site and adjust the field value by replacing one value with another). The syntax of the command can be seen below:

C:\>stsadm -help gl-importlistitem

stsadm -o gl-importlistitem

Imports a list item or items.

Parameters:
        -url <list view url to import into>
        -filename <import file name>
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-updateversions <1-3>
                1 - Add new versions to the current file (default)
                2 - Overwrite the file and all its versions (delete then insert)
                3 - Ignore the file if it exists on the destination]
        [-nofilecompression]
        [-quiet]
        [-retargetlinks (resets links pointing to the source to now point to the target)]
        [-sourceurl <url to a view of the original list> (use if retargetlinks)]
        [-retainobjectidentity]

Here's an example of how to import the items into the SiteDirectory list that we exported above and re-target any links pointing to the original documents so that they now point to the new documents (note that just like the built in import operation if you specify nofilecompression for the export you must also specify it for the import):

stsadm –o gl-importlistitem -url "http://teamsites/sitedirectory/siteslist/allitems.aspx" -filename "c:\SiteDirItems" -includeusersecurity -updateversions 2 -nofilecompression -retargetlinks -sourceurl "http://intranet/sitedirectory/siteslist/AllItems.aspx"

As with the exportlistitem operation a log file will be generated which will include all the details that were dumped to the console.

Update 11/15/2007: Apparently there's some sort of funky bug with the deployment API which is preventing the importing of list items from functioning 100% (see the comments below from David who discovered the issue). The problem is that if you import a list item that has dependencies in different folders either the dependencies or the dependent items will be placed in the wrong folder. There's a way with code that you can set the TargetParentUrl for orphaned items which is supposed to correct this - unfortunately it appears that whatever URL is first set gets used for all remaining items. The interesting thing is that you shouldn't need to set this property at all because all the list items are going to the same targe web and the WebUrl property of the SPImportSettings object is being set but if I don't set anything then the items still end up in the wrong location. The unfortunate side affect of this is that in order to prevent things from appearing in what would seem random locations and thus making it difficult to find imported items I've had to force everything to go to the root folder. Fortunately it's fairly easy to move things around after the import but it's still what should be an unnecessary pain in the butt. If anyone has any information which can help me solve this problem please pass it along (I spent two days picking through disassembled code trying to identify where the failure was but was not able to find anything conclusive - I also spent quite a bit of time trying to provide a work around to fix it but ran into to many issues where I'd fix one thing and break another (it's just too hard to predict every possible combination of possibilities that could occur)).

3. gl-copylistitem

If you've got the ability to import and export list items then why not make it a one step operation to copy a list item or move a list item. Once I had the code for the import and export the copy was nothing more than a wrapper which would call into the appropriate methods. By adding an extra parameter to support deleting the source list item I now also had a move command. The syntax of the command is just the combination of the import and export minus that which I don't need the user to set. The syntax of the command can be seen below:

C:\>stsadm -help gl-copylistitem

Copies a list item or items to another list.

Parameters:
        -sourceurl <list view url to copy>
        -targeturl <url of a web site to copy to>
        [-id <list item ID (separate multiple items with a comma)>]
        [-includeusersecurity]
        [-haltonwarning]
        [-haltonfatalerror]
        [-nologfile]
        [-versions <1-4>
                1 - Last major version for files and list items
                2 - The current version, either the last major or the last minor
                3 - Last major and last minor version for files and list items
                4 - All versions for files and list items (default)]
        [-updateversions <1-3>
                1 - Add new versions to the current file (default)
                2 - Overwrite the file and all its versions (delete then insert)

                3 - Ignore the file if it exists on the destination]
        [-quiet]
        [-retargetlinks (resets links pointing to the source to now point to the target)]
        [-deletesource]

Here's an example of how to perform the same operation that the above import and export commands were doing but with just one step while at the same time deleting the source list items:

stsadm –o gl-copylistitem -sourceurl "http://intranet/sitedirectory/siteslist/AllItems.aspx" -targeturl "http://teamsites/sitedirectory/siteslist/allitems.aspx" -id "1,2,3" -includeusersecurity -updateversions 2 -versions 4 -nofilecompression -retargetlinks -deletesource

If the source list items were deleted then the result of the export operation is saved to a temp folder, the path to which is displayed to the user upon completion. To delete all items in a list simply ommit the "id" parameter.

4. gl-deletelistitem

I created this command because I needed a quick way to clean up all my test items that I was creating while testing the gl-importlistitem command. So even though I didn't need it for my upgrade project specifically it came in handy for me so I figure someone else may benefit from it as well. The command allows you to either delete all items or specific items by passing in an "id" parameter. The syntax of the command can be seen below:

C:\>stsadm -help gl-deletelistitem

Deletes an item or items from a list.

Parameters:
        -url <list view URL>
        [-id <list item ID (separate multiple items with a comma)>]
        [-deletefolders]

Here's an example of how to delete items from a list:

stsadm –o gl-deletelistitem -url "http://intranet/sitedirectory/siteslist/AllItems.aspx" -id "1,2,3"

To delete all items from a list simply omit the "id" parameter.

Update 12/18/2007: I've added a deletefolders option to the gl-deletelistitem command. This parameter only takes affect when you don't specify an ID because you could previously delete a folder by specifying the folder's ID.

81 comments:

David Tappan said...

Great tool, but I had one issue: I exported some layout pages that had Preview Images associated with them, and the preview images were included in the export. When I imported them again to the same server (as a test), the preview images (e.g., articleleft.png) were created in the \masterpage folder, rather than in the \masterpage\en-us\Preview Images folder, which is where they came from. I checked the import log file, and it says it put the png files in the Preview Images folder, but it didn't; it put them in /masterpage. I checked the preview image in the CreatePage.aspx page, and they still work, but they point to the new location. So nothing broke, but I'm still concerned. Can you explain why it did this, and can it be fixed?

Gary Lapointe said...

If you look at the Manifest.xml file that gets generated when you do the export you'll see an ListItem node for the preview image. In there is the ParentFolderId attribute - looking at Microsoft's code it appears that this attribute is used to identify the folder but only when retainobjectidentity is true - it should be using the DirName attribute otherwise. I'll try to reproduce the problem and see if I can't at least repair it post import (the issue is definitely specific to Microsoft's code as I'm just using their Deployment API for all of this).

Gary Lapointe said...

David - the good news is that I was able to quickly reproduce the problem. The bad news is that despite two days of effort I was unable to find a work around or solution to it. There's definitely a bug in Microsoft's deployment API. I'm going to update the content above with my findings and send an email to Stefan Goßner to see if he has any thoughts. Thanks for pointing this out to me!

Gary Lapointe said...

David - I took another look at this and was able to provide a workaround which seems to do pretty well. It only works for files as there's no built-in way to move a list item that does not contain a file but I think this will make a big difference (it at least addresses the specific problem you identified). I've updated the code with my changes so feel free to download and give it a try.

Claudia said...

Gary, could you pls help to find out what could be causing the following error that happens during the process of importing list items using "gl-importlistitem"?

"...
Progress: Importing Folder /Empreendimentos/PublishingImages/Forms/Document.
FatalError: (null) "Empreendimentos/Lists/Empreendimentos/Forms/Document" not found.
at Microsoft.SharePoint.Library.SPRequest.CreateFolderOnImport(String bstrUrl, String bstrFolderU
rl, Guid guidFolderId, Int32 lDoclibRowId)
at Microsoft.SharePoint.Deployment.FolderSerializer.CreateFolder(SerializationInfoHelper infoHelp
er, SPWeb parentWeb, ImportObjectManager objectManager)
at Microsoft.SharePoint.Deployment.FolderSerializer.SetObjectData(Object obj, SerializationInfo i
nfo, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObjec
t)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChil
dObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()
Progress: Import Completed.
Finish Time: 5/7/2008 8:17:55 PM.
Completed with 0 warnings.
Completed with 1 errors.
"

It seems it's trying to create a folder called "Empreendimentos/Lists/Empreendimentos/Forms/Document" but I don't have such a folder in the original site from which I've exported the list items!

Tanzim Akhtar said...

Hi,

I am receiving the following error on using "gl-deleteitemlist".

Could not load file or assembly 'Microsoft.SharePoint.Publishing, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c' or one of its dependencies. The system cannot find the file specified.

However, I am able to use "gl-additemlist". What could be the possible reason?

Gary Lapointe said...

It's erroring out because you are probably running this against WSS which I've not tested on. The code is checking to see if the item you are deleting is the welcome page - I need to add some exception handling to account for when that dll does not exist (WSS) - I'll try to look at it this weekend if I have time.

Gary Lapointe said...

I just pushed a new build out which should resolve this issue for you.

Gary Lapointe said...

Claudia - unfortunately this is an issue with the content deployment API - I know that MSFT is working on a hotfix for the API but I don't have a timeline yet.

Gerald said...

Wonderful tool. My aim is to move one item from a discussion to another one. Unfortunately, the gl-copylistitem seems not to be capable to do this. No luck for me. Any idea ?

Gary Lapointe said...

Gerald - the copylistitem command will do what you are looking for - just pass in the ID of the item to copy.

Gerald said...

Thank you for your answser. I actually used the -id parameter (-id "1,2"). My test was done on WSS. 2 sample discussions board have been created.
The first one contains one dicussion (id=1) and a reply (id=2). The second one is empty. The operation generates no error when importing/exporting but displays a warning : WARNING: Unable to find parent folder for 'Lists/forum2/2_.000'.
Finally, the forum2 contains only the reply (without subject) but not the first discussion item.

Patrick said...

Doing some test with this I found it will not export survey items that are not completed. Is there anyway to get these as well?

Gary Lapointe said...

Patrick - I haven't done any testing with survey items so I'm not sure what's going on. Have you installed the latest content deployment rollup?

Patrick said...

I refreshed my verision of your extensions on 7/2, so I should have the latest version. I think that uncompleted survey entries may be hidden in the same way that new uploaded, unchecked in documents are. They only seem to be accessable only to the author.

Gary Lapointe said...

Gerald - it feels like a sequencing issue - like it's trying to add the reply before the original post. If you look at the exported manifext.xml file does it have item ID 1 listed first? As this is using the straight up content deployment stuff I'm honestly not sure what I can do to work around it. You might want to try the gl-exportlistitem2 and gl-addlistitem commands to see if they work any better for you.

Jason... said...

Great tool Gary...

Running the exportlist/importlistitem and coming across the issue of items not importing into the same list where we had exported (then deleted before importing again)...only doing this purely for testing purposes...

The importlistitem command was (live server/filenames excluded...)

stsadm -o gl-importlistitem -url "http://test1/testDocLib/Forms/AllItems.aspx" -filename "C:\testdoclib" -includeusersecurity -updateversions 2 -nofilecompression -retainobjectidentity

In the log file the following errors were occuring:

[7/29/2008 2:19:59 PM]: Start Time: 7/29/2008 2:19:59 PM.
[7/29/2008 2:19:59 PM]: Progress: Initializing Import.
[7/29/2008 2:20:00 PM]: Debug: Security check failed in OnUserGroupExport
[7/29/2008 2:20:00 PM]: Warning: The specified user could not be found.
*** Inner exception:
User cannot be found.
at Microsoft.SharePoint.SPUserCollection.FindUserSID(String strSearchText)
at Microsoft.SharePoint.SPUserCollection.GetBySID(String strSid)
at Microsoft.SharePoint.SPUserCollection.GetBySID(Byte[] byteSid)
at Microsoft.SharePoint.Deployment.SecurityObjectSerializer.ResolveUser(ImportStreamingContext context, SPWeb web, Byte[] sid, Int32& idUser)
[7/29/2008 2:20:00 PM]: Debug: Security check failed in OnUserGroupExport
[7/29/2008 2:20:00 PM]: Debug: Security check failed in OnUserGroupExport
[7/29/2008 2:20:00 PM]: Debug: Security check failed in OnUserGroupExport
[7/29/2008 2:20:00 PM]: Warning: The specified user could not be found.
.
.
.
[7/29/2008 3:17:34 PM]: Progress: Starting content import.
[7/29/2008 3:17:34 PM]: Progress: De-Serializing Objects to Database.
[7/29/2008 3:17:34 PM]: Progress: Importing File DeleteTestDocLib4/ConsultantList.udcx.
[7/29/2008 3:17:34 PM]: Progress: Importing File DeleteTestDocLib4/DoesFormExist.udcx.
[7/29/2008 3:17:34 PM]: Progress: Importing File DeleteTestDocLib4/GetEmailAddress.udcx.
[7/29/2008 3:17:34 PM]: Progress: Importing ListItem /DeleteTestDocLib4?id=1.
[7/29/2008 3:17:34 PM]: Debug: Security check failed in OnListItemImport
[7/29/2008 3:17:34 PM]: Warning: Access denied.
[7/29/2008 3:17:34 PM]: Progress: Importing ListItem /DeleteTestDocLib4?id=2.
[7/29/2008 3:17:34 PM]: Debug: Security check failed in OnListItemImport
[7/29/2008 3:17:34 PM]: Warning: Access denied.
[7/29/2008 3:17:34 PM]: Progress: Importing ListItem /DeleteTestDocLib4?id=3.
[7/29/2008 3:17:34 PM]: Debug: Security check failed in OnListItemImport
[7/29/2008 3:17:34 PM]: Warning: Access denied.
[7/29/2008 3:17:34 PM]: Debug: Security check failed in OnUserGroupExport
[7/29/2008 3:17:34 PM]: Progress: Import Completed.
[7/29/2008 3:17:34 PM]: Finish Time: 7/29/2008 3:17:34 PM.
[7/29/2008 3:17:34 PM]: Completed with 38 warnings.
[7/29/2008 3:17:34 PM]: Completed with 0 errors.

Gary, what causes these errors?

Gary Lapointe said...

Jason - most likely you have some users that were deleted from the site collection. The retains the IDs of users even if you delete them but when you try to reasign the users via the import it fails because it can't find them. Take a look at the items that you are trying to import and see what users it is looking for and then manually add those users to the site collection (or hack the exported files to point the items to a different user).

Jason said...

Thanks for the reply Gary...

I suppose the option I would select is adding the users back to the site collection (so long as they as still exist).

Does it actually matter if it is left as is?

Actually got this working (importing) with taking the retainobjectidentity parameter out but still the warning messages appearing.

The items are accessible (versioning and everything) so that is key.

The only issue I have had importing these item into a different farm (same site structure) where items were imported into the Root and not the Library designated for these list items. I understand this is a re-parenting issue and will update you once I have updated some code (using your importlistitem code as a base) to test this.

Thanks again Gary

massif said...

Hi Gary,

I'm new in Sharepoint techno.
Could you tell me how to install your STSADM extension ?
I put the file Lapointe.SharePoint.STSADM.Commands.WSS.wsp on the "web extensions" folder of the server but obviously there's other commands to do ... I'm looking for help in SP books but I'm sure your help can unblock me rapidly ! :-)

Gary Lapointe said...

First, make sure you have the correct WSP file - if your environment is MOSS download the other one otherwise you're good with that file. Next you need to run stsadm's addsolution and deploysolution commands to deploy my commands (if you download my source code there's a sample deployment batch file that you modify and use.

massif said...

Hi Gary,

deployment worked fine !
Thanks for having helped SP experts but novices too :-)

Anonymous said...

Great tool. More thanks for comments.

uli netzer said...

Hello SharePoint Admins!
These tools are great and make my life a lot easier. Thanks Gary.

My question: How do I get the IDs of the Listitems for an ExportListItem???

Gary Lapointe said...

The easiest way is to just modify the list view (or create a new one) and add in the ID column. Otherwise if you click to view the items properties you can find the ID in the URL.

guoguoer said...

Cool!
Thanks for your help :)

Ben Barkhouse said...

Gary,

I'm trying to use the export tool and it generating this error:
Violation of PRIMARY KEY constraint 'PK__#ExportObjects____7DE93661'. Cannot ins
ert duplicate key in object 'dbo.#ExportObjects'.
The statement has been terminated.

Any thoughts would be greatly appreciated!

Gary Lapointe said...

This is a common issue which I believe has been patched - see this thread: http://forums.technet.microsoft.com/en-US/sharepointadmin/thread/2e13998b-5cec-4201-b1b4-03b0d7b701c1/

Anonymous said...

I am trying to use deleteListitem on a document library to delete all the existing Infopath forms.
I am getting the error
Object reference not set to an instance of an object.

The Document LIbrary name has a space as well. In browser the path is something like Approval%20Docs.
Will deleteListitem work for document libraries with spaces in their name?

Gary Lapointe said...

Yes - it does work with lists with spaces. Just wrap the url in quotes.

oksana p said...

Gary, I wonder if you can help out with the issue. I downloaded and installed MOSS tool on a MOSS environment. The tool installed perfectly fine, but if I use gl-copylistitem on a set of id's, the tool fails during early import (no errors on export). After examining the logs, here is what I found:

Command ran: stsadm -o gl-copylistitem -sourceurl "http://server/subsite1/subsite2/Lists/CustomList/AllItems.aspx" -targeturl "http://server/subsite1/subsite2/Lists/CustomListArchive/AllItems.aspx" -id "2,3,4,5,6..." -includeusersecurity

e.g. what I am trying to accomplish is to copy a set of list items from one list on subsite2 to another list on the same subsite2.

Then the export log shows that the command is exporting a few items outside of subsite2, for example (from the log, names changed):
Exporting Folder /subsite1/subsite3/ReportingServicesReports/Forms/Document

On import, it attempts to import the "outside" folder into the subsite2, which fails. For example (from the log, names changed):
Progress: Importing Folder /subsite1/subsite2/ReportingServicesReports/Forms/Document

e.g. it takes a folder from a different subsite not specified in the gl-copylistitem command. The next entry in the log is the fatal error:

FatalError: (null) "subsite1/subsite2/Lists/CustomListArchive/Forms/Document" not found.
at Microsoft.SharePoint.Library.SPRequest.CreateFolderOnImport(String bstrUrl, String bstrFolderUrl, Guid guidFolderId, Int32 lDoclibRowId)
at Microsoft.SharePoint.Deployment.FolderSerializer.CreateFolder(SerializationInfoHelper infoHelper, SPWeb parentWeb, ImportObjectManager objectManager)
at Microsoft.SharePoint.Deployment.FolderSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector)
at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObject(Type objectType, Boolean isChildObject)
at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope)
at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream)
at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader)
at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects()
at Microsoft.SharePoint.Deployment.SPImport.Run()

Sorry for a long post, but any insight is appreciated.

Gary Lapointe said...

Oksana - This is the behavior of the content deployment API which is what I'm using (it attempts to copy and dependents that the items are dependent on). Try using the gl-exportlistitem and gl-importlistitem commands (which the copy command wraps) and set includedescendents to none. If that still doesn't work for you then try the gl-exportlistitem2 and gl-addlistitem commands (which don't use the content deployment API).

oksana p said...

Gary, thank you for a quick response. The gl-exportlistitem and gl-importlistitem exhibit the same behavior as mentioned earlier. The gl-exportitem2 and gl-addlistitem don't capture all of the versions, which is absolutely necessary in my case. Just an FYI - it appears that the gl-copylistitem works if I keep the number of copied items somewhere below 50 (sometimes it runs ok for 40 or so items, but definintely fails at 50), just wish there was a way to determine that "magic number" within API.

Bret Rowlinson said...

Gary,

Is it possible to use your migrate script (gl-copylistitem) to move items from a custom list into a form library? I am needing to do this but haven't found anything that can do it, your extensions look promising. I have installed your extension and ran the command, but get an error:

"Fatal Error: The given key was not present in the dictionary."... more content, I will post it if you want it.

Thanks!

Gary Lapointe said...

Bret - unfortunately it won't work to use to go from a standard custom list to a form (document) library. You might be able to do something with the gl-exportlistitem2/gl-addlistitem commands but I've never tried this scenario - most likely you'll have to write something custom.

Gary Lapointe said...

Oksana - was there any kind of stack trace or anything in the logs that's specific to the 50 item limit? I've typically only used this command in all of my testing to copy a very small number of items so I haven't seen this before.

Bret Rowlinson said...

Thanks Gary,

I was wondering if that was the case. I am surprised their isn't a way to handle this. The scenario I am looking at is importing an Excel file into Sharepoint (which then becomes a custom list), but then wanting to work with it using Infopath as the data entry/retrieval interface. I have been reading around about this, but have had no luck so far. Thank you for your extensions though, I am thinking they will prove quite useful as we start to use SharePoint more and more.

Cheers!

AlexMo said...

Gary,
Great tool but I am looking for a bit more precise definition of the -includedescendants flag, so I can understand what it does. I have a task list with related workflow history list. If I use the gl-copylistitems parameter and point it at the task list with -includedescendants specified, will it look into the workflow history, or do I have the "lineage" backwards (e.g., need to copy the history list and -includedescendants of that list)?

Gary Lapointe said...

See this SDK article about the SPIncludeDescendants enumeration: http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.deployment.spincludedescendants.aspx - basically, no - your items in the related list will not be copied.

John said...

no matter what I try to export, O get "Command line error."

here is a sample cmd line that fails:
stsadm –o gl-exportlistitem -url "http://localhost/sites/betaForms/Documents/Forms/AllItems.aspx" -id "1,2,3" -filename "c:\SiteDirItems" -includeusersecurity -versions 4 -nofilecompression

jdieter@twonails.com

Gary Lapointe said...

Try re-typing the command manually - looks like there's a special character for the hyphen.

Ruben Pillen said...

Gary,

I wanted to use your method copylistitem, as part of a disposition workflow, to move a certain publication page from a news-site to an archive-site in a publication environment with variations activated.

This can't be done without SPExportSettings.ExcludeDependencies property set to true, because without it, it will also try to move the corresponding page layout and Variation Relationship Item. We only want to move the Page itself.

Is it possible that you could add this property in your SPParamCollection parameters, so that we can use the -excludedependencies property in the commandline?
Thanks in advance.

BTW. I also noticed that you can't use the -nofilecompression in gl-copylistitem which you do refer to in your example.

Gary Lapointe said...

Ruben - thanks for the feedback - I went ahead and added those parameters (slight oversight on my part - they should have been there) - I'll try to get the code deployed sometime tonight.

Anonymous said...

Hi Gary,
Thank you very much, its working fine with List/Libraries.
When i try gl-copylistitem for Discussion boards its is giving error as "Either List contains no items or the list item cannot be found."

Could you please let me how can we use this command for Discussion boards.
ITs very urgent, please help me .

Thanks in advance
Meera

Gary Lapointe said...

Discussion items are kind of odd in the way that they use folder items to store threads. Try using the gl-exportlist/gl-importlist instead of the copy and play around with the retainobjectidentity and the includedependencies parameters

JC said...

Hi.
Great job!
I have issue: when i export/import
a wiki document library, the [[links]] in the pages don't keep their "[[ ]]".
Did I miss a parameter ?

Gary Lapointe said...

I've not tested with wiki libraries but all i'm doing is using the out-of-the-box content migration api. What version of the product are you using? Make sure you at least have the infrastructure update installed.

Anonymous said...

Hi Gary,

I'm getting this error when I use this command:

[05/02/2009 17:15:51]: Heure de début : 05/02/2009 17:15:51.
[05/02/2009 17:15:51]: Progress: Initialisation de l'opération Importer.
[05/02/2009 17:15:52]: Progress: Début d'importation du contenu.
[05/02/2009 17:15:52]: FatalError: L'élément a déjà été ajouté. Clé du dictionnaire : 'fpg-ce-numér rmp-081110.xls' Clé ajoutée : 'fpg-ce-numér rmp-081110.xls'
à System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
à System.Collections.Hashtable.Add(Object key, Object value)
à System.Collections.Specialized.StringDictionary.Add(String key, String value)
à Lapointe.SharePoint.STSADM.Commands.Lists.ImportListItem.OnImportStarted(Object sender, SPDeploymentEventArgs e)
à Microsoft.SharePoint.Deployment.SPDeployment.OnStarted(SPDeploymentEventArgs e)
à Microsoft.SharePoint.Deployment.SPImport.Run()
[05/02/2009 17:15:53]: Progress: Opération Importer terminée.
[05/02/2009 17:15:53]: Heure de fin : 05/02/2009 17:15:53.
[05/02/2009 17:15:53]: Opération terminée : 0 avertissements.
[05/02/2009 17:15:53]: Opération terminée : 1 erreurs.


in English : Item has already been added !!

Works when I delete the file of course.

Any ideas?

Bob said...

I am using gl-copylistitem to copy calendar items to another site's calendar, but when I copy them, I seem to get double entries in the new calendar. Is this anything anyone else has experienced or perhaps an issue with my original data?

Gary Lapointe said...

Bob - I'm not really sure why you'd be getting duplicate items as I've not seen that before.

LC said...

Gary hi,

I created a publishing site with variations. I wanted to duplicate a site, but as you probably know the option of "save site as template" doesn't work with publishing sites.
I created a publishing site, and with your GREAT solution of importlistitem I exported pages from the Pages Library and imported them to a new site. But the variation created only some of the items. Do you know what the problem may be?

Thanks,
Armine.

Gary Lapointe said...

The hidden relationships list is out of sync - see my "Fun With Variations" post about the list - I have a command that will repair the list but I've done any testing with subsites (I've got a link to another tool that someone else developed that should work for you - it's either on the variations post or the fixvariationrelationships command post).

L.C. said...

Yes, I've tried the Variations Editor and te fixrelationship....but it didn't help. The problem is that when i import pages to the source label(variations) it's supposed to duplicate them in my other label, it only duplicates 3 of 5 or 1 of 5. It's weird because I didn't touch the relationship list(and the fix doesn't help). I created the website and then I tried to import the pages...

Alex @ SMS said...

Hi Gary,
I am having problem installing the WSS msi. I am getting an Error 1001. I have MOSS environement which I need to administrate. I am trying to use "gl-delelelist" cmd and no luck. Please assist
Alex S

Gary Lapointe said...

Are you installing the PowerShell MSI file or just the stsadm command WSP file? Did you successfully deploy the WSP? Are you able to get the help for the command?

Paul said...

Not sure exactly what I'm doing wrong, but I'm using your export command like this:

stsadm -o gl-exportlistitem -url "https://sourceurl/AllItems.aspx" -filename "c:\myfile.bak" -versions 4 -includeusersecurity -id "115,120"

and it appears to work without a problem. Then I run the import like this:

stsadm -o gl-importlistitem -url "https://targeturl" -filename "c:\myfile.bak" -includeusersecurity -updateversions 3

and it tries to copy the items back to the same library (over the same items) that the backup was made from (I know this because I ran the command with the updateversions param set to 2 and it wound up readding all the document versions onto the existing document, so now I have twice as many versions as I originally had). Is it possible that this command is ignoring the target url? I've downloaded your source code and am going to pick through it, but if you have a chance to respond I'd appreciate it.

Alex @ SMS said...

It tried both... the MSI did not work and neither the deploy is said it could not find the file. Even if the wps file is in the same directory. the message is "wsp does not exist in the solution store". I am not sure what do next..I am novice stsadm but am a developer. thanks

Gary Lapointe said...

You need to add the WSP to the store by using addsolution, then use deploysolution.

Gary Lapointe said...

Paul - the only way that would happen is if you have a hosts file entry or DNS entry that is pointing your target URL to your source URL. I'm definitely not ignoring the target URL (I use this command all the time and I've never experienced the issue you are describing).

Scott said...

Gary - I can't tell you how often I've used your stsadm extensions. THANK YOU!!!

Today I wanted to clean up some test items and when I ran the command:

stsadm -o gl-deletelistitem -url http://sharepoint/images/forms/allitems.aspx

Here is the result:

Object reference not set to an instance of an object.

Any ideas?

Supporting Artifact:
- I just upgraded to SP2 today.
- I've never used this command prior to today. So I can't say this worked yesterday.
- I downloaded the latest wsp file today to verify I've not missed anything updates.
- MOSS 32bit installation / Windows Server 2008

If you have any suggestions please let me know.

Thanks in advance!

Scott

Gary Lapointe said...

Scott - not sure why you were getting the error - I went ahead and added some additional null checks (shouldn't be necessary but as I don't know what's causing the error...). I'll push out an updated build tonight - if you could try again I'd appreciate any feedback you can provide.

MaxPower555 said...

Hey Mate,
Great tool. But Im trying to copy a list item to another list but both have lookup fields and I keep getting the error.
Warning: Unable to find a lookup list /. The list is not part of the exported package and does not exist in the destination site collection.

I have all the lookup fields in place but it still doesnt want to work, I was wondering if there is a work around for this?

Many Thanks,

MaxPower555 said...

Oh btw, here is the command line im using

stsadm -o gl-copylistitem -sourceurl "http://server/sites/test/Lists/RFC%2
0Live/AllItems.aspx" -targeturl "http://server/sites/test/Lists/RFC1/AllItems.
aspx" -id "1,2" -includeusersecurity -updateversions 2 -nofilecompression -retargetlinks

Nathan said...

Hi Gary!

i try the latest build and still have the same error with Scott.

i execute this command :
stsadm -o gl-deletelistitem -url "http://portal/sites/sitename/Document Library Name/Forms/AllItems.aspx" -deletefolders

and it returned the same error :
Object reference not set to an instance of an object.

i use your latest build in this environtment :
- MOSS 2007 SP2
- 64 bit architecture
- Windows server 2008

any solution?

Thanks alot for your great help!

Bas said...

Hi Gary,

I've received an error using gl-importlistitem command: "FatalError: Invalid file name. The file name you specified could not be used. It may be the name of an existing file or directory, or you may not have permission to access the file".

After a lot of debugging, it turned out that this error was caused by a case sensitive URL. Unlike using an URL with different casing while browsing is not relevant, the SPImport.Run() method apparently requires that the URL to the list view is exactly the same as the actual SiteURL, that is: case sensitive.

After passing the correct address to the list view URL parameter, the gl-importlistitem command worked like a charm.

If your findings are the same as mine, maybe you can add a comment to the -url parameter regarding this strange behavior. I hope this information is useful for some of your blog readers.

With kind regards from the Netherlands.

Emerson said...

I can import data from .xlsx or xls...??? Because i try everything, and i can´t do that...??? Please i need help.
Emerson - Brasil

Gary Lapointe said...

None of my stuff supports importing data from an excel file. You can try writing something that reads the file (or convert to csv and use powershell) and then call my gl-addlistitem command.

Anonymous said...

Hi Gary, thankyou for your deserving committment with stsadm and Sharepoint Content Deployement!

Can you give me some advice about following "strange" problem? I want to copy list "Rilasci" from /sgq/pnxmi to /sgq/pnxmi20

I've tried using:
stsadm -o gl-copylistitem -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -targeturl "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -nofilecompression -retargetlinks -includeusersecurity

0) the log says
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi20/Lists/Rilasci?id=1.
1) items ARE imported
2) items does not retains object identity (no options for that)
3) lookup fields continue to point to the original site (/sgq/pnxmi)

so i've tried this:
stsadm -o gl-exportlistitem -url "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -overwrite

stsadm -o gl-importlistitem -url "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -retainobjectidentity -retargetlinks -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx"

BUT items ARE NOT imported at all and the target list remains empty

Log says that
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi/Lists/Rilasci?id=1.

(this is the name of the source item)

Do you have any comment? Do you see something wrong?

thank you in advance for any help
Roberto

Anonymous said...

Hi Gary, thankyou for your deserving committment with stsadm and Sharepoint Content Deployement!

May be you can give me some advice about this "strange" problem: I simply want to copy list "Rilasci" from /sgq/pnxmi to /sgq/pnxmi20

I've tried using this: stsadm -o gl-copylistitem -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -targeturl "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -nofilecompression -retargetlinks -includeusersecurity

0) the log says that
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi20/Lists/Rilasci?id=1.
1) items ARE imported
2) items does not retains object identity (no options for that)
3) lookup fields continue to point to the original site (/sgq/pnxmi)

so i've tried this: stsadm -o gl-exportlistitem -url "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -overwrite
stsadm -o gl-importlistitem -url "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -retainobjectidentity -retargetlinks -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx"

0) items ARE NOT imported at all and the target list remains empty
1) Log says that :
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi/Lists/Rilasci?id=1.

(this is the name of the SOURCE item, while the log of gl-copylistitem refers to the target)

Do you see something wrong? Do you have any comment/suggestion?

thank you in advance for any help
Roberto

Roberto said...

Hi Gary, thankyou for your deserving committment with stsadm and Sharepoint Content Deployement!

May be you can give me some advice about this problem importing list items : I simply want to copy my list named "Rilasci" from /sgq/pnxmi to /sgq/pnxmi20

I've tried using this: stsadm -o gl-copylistitem -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -targeturl "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -nofilecompression -retargetlinks -includeusersecurity

0) the log says that
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi20/Lists/Rilasci?id=1.
1) pls note pnxmi20 is the name of the target url. correct
2) items ARE imported
3) items does not retains object identity (no options for that)

So i've tried this: stsadm -o gl-exportlistitem -url "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -overwrite
stsadm -o gl-importlistitem -url "https://garuda:32008/sgq/pnxmi20/Lists/Rilasci/AllItems.aspx" -filename "C:\Temp\Export\pnx\Rilasci" -includeusersecurity -nofilecompression -retainobjectidentity -retargetlinks -sourceurl "https://garuda:32008/sgq/pnxmi/Lists/Rilasci/AllItems.aspx"

0) items ARE NOT imported at all and the target list remains empty
1) Log says that :
[28/09/2009 13.07.04]: Progress: È in corso l'importazione della voce di elenco /sgq/pnxmi/Lists/Rilasci?id=1.
2) pls note pnxmi is the name of the SOURCE url. INCORRECT

Do you see something wrong? Do you have any comment/suggestion?

thank you in advance for any help
Roberto

Anonymous said...

I seem to be having problems copying across anything within a hyperlink field, they just don't come acorss at all, do you know if this should work or not.

Awesome scripts by the way, I use them heaps!!

Anonymous said...

Ignore my previous query about hyperlinks not coming across, deleting the list and re-adding it fixed the problem, weird, but it's working now.

Larry W. Virden said...

Hello
I have what I hope is a question that is simple for you to answer.
I'm working on a MOSS 2007 system.

I tried out your gl-copylistitem , attempting to move a versioned file from one doc library to another doc library in the same site.

The output of your command says that there were no warnings or errors. The file is no longer at the source location.

However, the file does not appear in the doc library specified as the target. When I look at the entire site's contents, the file isn't shown anywhere on the site.

I notice in this blog's update dated 11/15/2007 you talk about a situation caused by a Microsoft api bug that might cause unexpected results. In my case, there were no known dependencies - this was a simple word document, no images, just created by me to test out the move functionality.

So my question is this - how can I determine if the file still exists, and if so, how would I access it to move it where it should be?

Gary Lapointe said...

Larry - I've recently had a few posts from people stating that the imports are not working (one or more items don't show up) - I've not been able to reproduce it but I suspect it's related to one of the CUs, I just don't know which one. Ultimately I'm just wrapping the content deployment API which is doing all the work - all I do is set some properties and run it. The exported file should be in your temp folder so it should be possible to try the import again. I'd suggest digging through your ULS logs to see if there's any other information that might explain why it reported success but didn't work.

Anonymous said...

Great tool , but i am seeing issue with gl-copylistitem , gl-importlistitem, gl-exportlistitem , the tool exports all files except files of type .txt . the behaviour is consistent with
gl-copylistitem
gl-exportlistitem
gl-importlistitem
Any idea or suggestion .

Rob Ellis said...

I'm also having problems with the delete option - getting Object reference not set to an instance of an object.

Running MOSS SP2 and Dec 09 CU, on x64 Windows 2003 R2 SP2.

Rob

Rob Ellis said...

As a follow up to my other comment - I've found that I did not have permissions to delete the list items, which would explain the error I was seeing.

Rob Ellis said...

As another follow-up - I've sorted the permissions, but I'm still getting the error.

I have tried a couple of different site collections, creating new libraries, etc, but the problem is the same regardless.

Goran Obradovic said...

Hello!

Great tool, but most usable (for me) command - gl-copylistitem, does not work properly. Actually, command works, but it is weird problem already mentioned in two of posts here - some items are missing. I believe that no one here determined that items ARE successfully imported, but they go missing after second or so of being in destination list. I managed to determine this weirdness when I copied all items in one large list, and watched destination as import was progressing. All items were imported and after ten seconds or so, only part of them (1/4 or so) was left in list. Each item that was left could be copied individually, and the ones missing cannot be copied when ID is passed (that is, they ARE copied, but deleted immediately).

Hope that this helps to determine cause.

Ed. said...

This doesn't appear to work on Issues lists, only on Document Libraries. Is that true? I have a need to merge several Issues lists into one and I thought gl-copylistitem would be the perfect tool. Thanks.
- Ed.

J said...

Pity, but this won't work for items in recurring meeting workspace that are NOT upcoming meeting items :(

Don't know what methods are for migration api to handle this situation, but for using SPQuery, it's a matter of setting SPQuery.MeetingInstanceId to -1