Skip to content

Commit 671eed8

Browse files
kill74kill74
authored
Create README.md
1 parent a544827 commit 671eed8

1 file changed

Lines changed: 193 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
SharePoint Sync Tool
2+
https://img.shields.io/badge/.NET%2520Framework-4.8-blue
3+
https://img.shields.io/badge/SharePoint%2520CSOM-16.1-green
4+
https://img.shields.io/badge/License-MIT-yellow.svg
5+
6+
A console application to synchronize SharePoint Online lists with a SQL Server database, including data quality routines for common list inconsistencies.
7+
Built with the SharePoint Client Side Object Model (CSOM) and .NET Framework 4.8.
8+
9+
Table of Contents
10+
Overview
11+
12+
Features
13+
14+
Prerequisites
15+
16+
Installation
17+
18+
Configuration
19+
20+
Usage
21+
22+
Project Structure
23+
24+
Troubleshooting
25+
26+
License
27+
28+
Português
29+
30+
Overview
31+
This tool was developed to automate the transfer of data from SharePoint Online lists to SQL Server tables, as well as to perform targeted data quality fixes on specific lists (e.g., Activities, Invoice Request). It supports two sync modes (daily and monthly) and provides detailed logging for monitoring and debugging.
32+
33+
Features
34+
SharePoint → SQL Synchronisation
35+
Transfers data from configured SharePoint lists to corresponding SQL Server tables using either incremental (daily) or full (monthly) updates.
36+
37+
Data Quality Fixes
38+
39+
Activities: Backfills the _OpportunityID field by copying the value from OpportunityID where it is null.
40+
41+
Invoice Request: Updates approver fields (Main_x0020_approver, Optional_x0020_approver, Financial_x0020_approver) based on recent changes in the Unit list.
42+
43+
Timesheet: (similar data quality logic – extend as needed)
44+
45+
SQL Connection Testing
46+
Before any operation, the tool verifies connectivity to the SQL Server and checks for necessary permissions (SELECT, CREATE TABLE, etc.).
47+
48+
Configurable Logging
49+
Verbosity levels (0–3) control the amount of console output, making it easy to run silently or in debug mode.
50+
51+
External Configuration
52+
All settings (SharePoint credentials, SQL connection string) are stored in an XML file, which can be overridden via command-line argument.
53+
54+
Prerequisites
55+
Windows operating system (the application targets .NET Framework 4.8)
56+
57+
.NET Framework 4.8 (or later) – Download
58+
59+
Visual Studio 2019 or 2022 (for building from source)
60+
61+
Access to a SharePoint Online tenant with appropriate permissions to read/write lists
62+
63+
SQL Server (any edition) with a database where the target tables will be created/updated
64+
65+
VPN if your SharePoint or SQL Server is only accessible within a corporate network
66+
67+
Installation
68+
Clone the repository
69+
70+
bash
71+
git clone https://github.com/your-org/sharepoint-sync-tool.git
72+
Open the solution in Visual Studio
73+
ConsoleApp1.sln
74+
75+
Restore NuGet packages
76+
The main package is Microsoft.SharePointOnline.CSOM (version 16.1).
77+
Visual Studio should restore it automatically; if not, run:
78+
79+
text
80+
Update-Package -reinstall
81+
Build the solution (Build → Build Solution)
82+
The executable will be placed in bin\Debug or bin\Release.
83+
84+
(Optional) Publish – you can copy the bin\Release folder to any Windows machine with .NET Framework 4.8 installed.
85+
86+
Configuration
87+
All runtime settings are defined in an XML file. The default location is XmlConfig\UserConfig.xml (relative to the executable). You can specify a different file with the --config argument.
88+
89+
Example UserConfig.xml
90+
xml
91+
<?xml version="1.0" encoding="utf-8"?>
92+
<Configuration>
93+
<SharePoint>
94+
<Username>user@company.com</Username>
95+
<Password>YourPassword</Password>
96+
</SharePoint>
97+
<SQL>
98+
<ConnectionString>Server=myServer;Database=myDB;Integrated Security=true;</ConnectionString>
99+
</SQL>
100+
</Configuration>
101+
Security Note: Storing passwords in plain text is not recommended for production. Consider using encrypted configuration sections or environment variables.
102+
103+
Configuration Elements
104+
Element Description
105+
SharePoint/Username The SharePoint Online user account (email format).
106+
SharePoint/Password The password for that account.
107+
SQL/ConnectionString A valid ADO.NET connection string to your SQL Server database.
108+
The tool expects specific SharePoint site relative paths (e.g., seed, wolf, selfservice/invoicerequest) – these are hard-coded in the data quality classes. If your site structure differs, you will need to adjust the source code.
109+
110+
Usage
111+
Run the executable from the command line:
112+
113+
text
114+
ConsoleApp1.exe [arguments]
115+
Arguments
116+
Argument Description
117+
daily Perform incremental sync (typically only new/changed items).
118+
monthly Perform a full sync (all items).
119+
diagnostic Enable diagnostic mode (sets verbosity to 1 if not otherwise specified).
120+
--verbose=<0-3> Set verbosity level: 0 = quiet, 1 = normal, 2 = detailed, 3 = very detailed (debug).
121+
--config=<path> Use an alternative configuration file.
122+
Examples
123+
bash
124+
# Daily sync with normal logging, default config
125+
ConsoleApp1.exe daily
126+
127+
# Monthly sync with detailed logging and custom config
128+
ConsoleApp1.exe monthly --verbose=2 --config="C:\Configs\custom.xml"
129+
130+
# Just test SQL connection and exit (diagnostic mode)
131+
ConsoleApp1.exe diagnostic
132+
What happens during execution?
133+
The tool parses command-line arguments and sets up logging.
134+
135+
It tests the SQL Server connection (including permission checks).
136+
137+
SharePoint credentials are retrieved from the configuration.
138+
139+
Based on the mode (daily or monthly), it calls RefreshSQLLists.SPOtoSQLUpdate(...) which performs the actual data transfer.
140+
141+
Data quality routines (ActivitiesDQ, InvoiceRequestDQ, etc.) are executed as part of the sync process.
142+
143+
All actions are logged to the console according to the verbosity level.
144+
145+
Project Structure
146+
text
147+
ConsoleApp1/
148+
├── ConsoleApp1.csproj
149+
├── packages.config
150+
├── AssemblyInfo.cs
151+
├── ConsoleLogger/
152+
│ └── Logger.cs
153+
├── Sharepoint/
154+
│ ├── ActivitiesDQ.cs
155+
│ ├── Context.cs
156+
│ ├── GetallLists.cs
157+
│ ├── InvoiceRequestDQ.cs
158+
│ ├── SPOList.cs
159+
│ ├── SPOUser.cs
160+
│ └── TimesheetDQ.cs
161+
├── SPODataQuality/
162+
│ └── RefreshSPOLists.cs
163+
├── Sqlserver/
164+
│ ├── RefreshSQLLists.cs
165+
│ └── SQLInteraction.cs
166+
└── XmlConfig/
167+
├── ConfigHelper.cs
168+
└── UserConfig.xml
169+
Key components:
170+
171+
RefreshSPOLists – main entry point (Main method), argument handling, SQL test, and orchestration.
172+
173+
SPOList / SPOUser – wrappers for SharePoint CSOM operations.
174+
175+
ActivitiesDQ / InvoiceRequestDQ – data quality fixes for specific lists.
176+
177+
RefreshSQLLists – handles the actual SharePoint-to-SQL data transfer.
178+
179+
ConfigHelper – reads the XML configuration file.
180+
181+
Logger – simple console logger with verbosity levels.
182+
183+
Troubleshooting
184+
Problem Possible Solution
185+
Cannot connect to SQL Server Check VPN, firewall rules, and the connection string. Run diagnostic mode for more details.
186+
SharePoint login fails Verify username/password in config. Ensure the account has access to the specified site.
187+
"Field not found" errors The data quality classes expect specific field names (e.g., OpportunityID). If your lists use different names, you must modify the code.
188+
Throttling / slow performance The tool batches updates (e.g., 80 items per batch) to avoid SharePoint limits. Adjust batch sizes if needed.
189+
Missing tables in SQL The tool assumes tables already exist with the correct schema. Review SQLInteraction.cs for table creation logic.
190+
If you encounter unexpected behaviour, run with --verbose=3 to get detailed debug output and open an issue with the log.
191+
192+
License
193+
This project is licensed under the MIT License – see the LICENSE file for details.

0 commit comments

Comments
 (0)