Skip to content

Commit 6e2e943

Browse files
kill74kill74
authored
Update README.md
1 parent 671eed8 commit 6e2e943

1 file changed

Lines changed: 131 additions & 106 deletions

File tree

‎README.md‎

Lines changed: 131 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,93 +1,100 @@
1-
SharePoint Sync Tool
2-
https://img.shields.io/badge/.NET%2520Framework-4.8-blue
3-
https://img.shields.io/badge/SharePoint%2520CSOM-16.1-green
4-
https://img.shields.io/badge/License-MIT-yellow.svg
1+
# SharePoint Sync Tool
52

6-
A console application to synchronize SharePoint Online lists with a SQL Server database, including data quality routines for common list inconsistencies.
7-
Built with the SharePoint Client Side Object Model (CSOM) and .NET Framework 4.8.
3+
![.NET Framework](https://img.shields.io/badge/.NET%20Framework-4.8-blue)
4+
![SharePoint CSOM](https://img.shields.io/badge/SharePoint%20CSOM-16.1-green)
5+
![License](https://img.shields.io/badge/License-MIT-yellow.svg)
86

9-
Table of Contents
10-
Overview
7+
A console application to synchronise SharePoint Online lists with a SQL Server database, including data quality routines for common list inconsistencies. Built with the SharePoint Client Side Object Model (CSOM) and .NET Framework 4.8.
118

12-
Features
9+
---
1310

14-
Prerequisites
11+
## Table of Contents
1512

16-
Installation
13+
- [Overview](#overview)
14+
- [Features](#features)
15+
- [Prerequisites](#prerequisites)
16+
- [Installation](#installation)
17+
- [Configuration](#configuration)
18+
- [Usage](#usage)
19+
- [Project Structure](#project-structure)
20+
- [Troubleshooting](#troubleshooting)
21+
- [License](#license)
1722

18-
Configuration
23+
---
1924

20-
Usage
25+
## Overview
2126

22-
Project Structure
27+
This tool automates the transfer of data from SharePoint Online lists to SQL Server tables and performs targeted data quality fixes on specific lists (e.g. Activities, Invoice Request). It supports two sync modes — **daily** and **monthly** — and provides detailed logging for monitoring and debugging.
2328

24-
Troubleshooting
29+
---
2530

26-
License
31+
## Features
2732

28-
Português
29-
30-
Overview
31-
This tool was developed to automate the transfer of data from SharePoint Online lists to SQL Server tables, as well as to perform targeted data quality fixes on specific lists (e.g., Activities, Invoice Request). It supports two sync modes (daily and monthly) and provides detailed logging for monitoring and debugging.
32-
33-
Features
34-
SharePoint → SQL Synchronisation
33+
**SharePoint → SQL Synchronisation**
3534
Transfers data from configured SharePoint lists to corresponding SQL Server tables using either incremental (daily) or full (monthly) updates.
3635

37-
Data Quality Fixes
36+
**Data Quality Fixes**
37+
- *Activities* — Backfills the `_OpportunityID` field by copying from `OpportunityID` where null.
38+
- *Invoice Request* — Updates approver fields (`Main_x0020_approver`, `Optional_x0020_approver`, `Financial_x0020_approver`) based on recent changes in the Unit list.
39+
- *Timesheet* — Similar data quality logic, extendable as needed.
3840

39-
Activities: Backfills the _OpportunityID field by copying the value from OpportunityID where it is null.
41+
**SQL Connection Testing**
42+
Verifies connectivity to SQL Server and checks for necessary permissions (SELECT, CREATE TABLE, etc.) before any operation begins.
4043

41-
Invoice Request: Updates approver fields (Main_x0020_approver, Optional_x0020_approver, Financial_x0020_approver) based on recent changes in the Unit list.
44+
**Configurable Logging**
45+
Verbosity levels (0–3) control console output — run silently in production or in full debug mode during development.
4246

43-
Timesheet: (similar data quality logic – extend as needed)
47+
**External Configuration**
48+
All settings (SharePoint credentials, SQL connection string) are stored in an XML file, overridable via a command-line argument.
4449

45-
SQL Connection Testing
46-
Before any operation, the tool verifies connectivity to the SQL Server and checks for necessary permissions (SELECT, CREATE TABLE, etc.).
50+
---
4751

48-
Configurable Logging
49-
Verbosity levels (0–3) control the amount of console output, making it easy to run silently or in debug mode.
52+
## Prerequisites
5053

51-
External Configuration
52-
All settings (SharePoint credentials, SQL connection string) are stored in an XML file, which can be overridden via command-line argument.
54+
- Windows operating system (.NET Framework 4.8 target)
55+
- [.NET Framework 4.8](https://dotnet.microsoft.com/download/dotnet-framework/net48) or later
56+
- Visual Studio 2019 or 2022 (for building from source)
57+
- Access to a SharePoint Online tenant with appropriate read/write permissions
58+
- SQL Server (any edition) with a database where target tables will be created/updated
59+
- VPN access if SharePoint or SQL Server is only reachable within a corporate network
5360

54-
Prerequisites
55-
Windows operating system (the application targets .NET Framework 4.8)
61+
---
5662

57-
.NET Framework 4.8 (or later) – Download
63+
## Installation
5864

59-
Visual Studio 2019 or 2022 (for building from source)
65+
**1. Clone the repository**
66+
```bash
67+
git clone https://github.com/your-org/sharepoint-sync-tool.git
68+
```
6069

61-
Access to a SharePoint Online tenant with appropriate permissions to read/write lists
70+
**2. Open the solution in Visual Studio**
71+
```
72+
ConsoleApp1.sln
73+
```
6274

63-
SQL Server (any edition) with a database where the target tables will be created/updated
75+
**3. Restore NuGet packages**
6476

65-
VPN if your SharePoint or SQL Server is only accessible within a corporate network
77+
The main dependency is `Microsoft.SharePointOnline.CSOM` (v16.1). Visual Studio should restore it automatically. If not, run:
78+
```
79+
Update-Package -reinstall
80+
```
6681

67-
Installation
68-
Clone the repository
82+
**4. Build the solution**
6983

70-
bash
71-
git clone https://github.com/your-org/sharepoint-sync-tool.git
72-
Open the solution in Visual Studio
73-
ConsoleApp1.sln
84+
Go to *Build → Build Solution*. The executable will be placed in `bin\Debug` or `bin\Release`.
7485

75-
Restore NuGet packages
76-
The main package is Microsoft.SharePointOnline.CSOM (version 16.1).
77-
Visual Studio should restore it automatically; if not, run:
86+
**5. (Optional) Deploy**
7887

79-
text
80-
Update-Package -reinstall
81-
Build the solution (Build → Build Solution)
82-
The executable will be placed in bin\Debug or bin\Release.
88+
Copy the `bin\Release` folder to any Windows machine with .NET Framework 4.8 installed.
89+
90+
---
8391

84-
(Optional) Publish – you can copy the bin\Release folder to any Windows machine with .NET Framework 4.8 installed.
92+
## Configuration
8593

86-
Configuration
87-
All runtime settings are defined in an XML file. The default location is XmlConfig\UserConfig.xml (relative to the executable). You can specify a different file with the --config argument.
94+
All runtime settings are defined in an XML file. The default location is `XmlConfig\UserConfig.xml` (relative to the executable). You can specify an alternative path with the `--config` argument.
8895

89-
Example UserConfig.xml
90-
xml
96+
**Example `UserConfig.xml`**
97+
```xml
9198
<?xml version="1.0" encoding="utf-8"?>
9299
<Configuration>
93100
<SharePoint>
@@ -98,52 +105,65 @@ xml
98105
<ConnectionString>Server=myServer;Database=myDB;Integrated Security=true;</ConnectionString>
99106
</SQL>
100107
</Configuration>
101-
Security Note: Storing passwords in plain text is not recommended for production. Consider using encrypted configuration sections or environment variables.
108+
```
102109

103-
Configuration Elements
104-
Element Description
105-
SharePoint/Username The SharePoint Online user account (email format).
106-
SharePoint/Password The password for that account.
107-
SQL/ConnectionString A valid ADO.NET connection string to your SQL Server database.
108-
The tool expects specific SharePoint site relative paths (e.g., seed, wolf, selfservice/invoicerequest) – these are hard-coded in the data quality classes. If your site structure differs, you will need to adjust the source code.
110+
> ⚠️ **Security Note:** Storing passwords in plain text is not recommended for production. Consider using encrypted configuration sections or environment variables.
111+
112+
| Element | Description |
113+
|---|---|
114+
| `SharePoint/Username` | The SharePoint Online user account (email format) |
115+
| `SharePoint/Password` | The password for that account |
116+
| `SQL/ConnectionString` | A valid ADO.NET connection string to your SQL Server database |
117+
118+
> **Note:** The tool expects specific SharePoint site relative paths (e.g. `seed`, `wolf`, `selfservice/invoicerequest`). These are hard-coded in the data quality classes. If your site structure differs, adjust the source code accordingly.
119+
120+
---
121+
122+
## Usage
109123

110-
Usage
111124
Run the executable from the command line:
112125

113-
text
126+
```
114127
ConsoleApp1.exe [arguments]
115-
Arguments
116-
Argument Description
117-
daily Perform incremental sync (typically only new/changed items).
118-
monthly Perform a full sync (all items).
119-
diagnostic Enable diagnostic mode (sets verbosity to 1 if not otherwise specified).
120-
--verbose=<0-3> Set verbosity level: 0 = quiet, 1 = normal, 2 = detailed, 3 = very detailed (debug).
121-
--config=<path> Use an alternative configuration file.
122-
Examples
123-
bash
124-
# Daily sync with normal logging, default config
128+
```
129+
130+
### Arguments
131+
132+
| Argument | Description |
133+
|---|---|
134+
| `daily` | Perform incremental sync (new/changed items only) |
135+
| `monthly` | Perform a full sync (all items) |
136+
| `diagnostic` | Enable diagnostic mode (sets verbosity to 1 if not otherwise specified) |
137+
| `--verbose=<0-3>` | Set verbosity: `0` = quiet, `1` = normal, `2` = detailed, `3` = debug |
138+
| `--config=<path>` | Use an alternative configuration file |
139+
140+
### Examples
141+
142+
```bash
143+
# Daily sync with normal logging
125144
ConsoleApp1.exe daily
126145

127-
# Monthly sync with detailed logging and custom config
146+
# Monthly sync with detailed logging and a custom config
128147
ConsoleApp1.exe monthly --verbose=2 --config="C:\Configs\custom.xml"
129148

130-
# Just test SQL connection and exit (diagnostic mode)
149+
# Test SQL connection and exit
131150
ConsoleApp1.exe diagnostic
132-
What happens during execution?
133-
The tool parses command-line arguments and sets up logging.
134-
135-
It tests the SQL Server connection (including permission checks).
151+
```
136152

137-
SharePoint credentials are retrieved from the configuration.
153+
### What happens during execution?
138154

139-
Based on the mode (daily or monthly), it calls RefreshSQLLists.SPOtoSQLUpdate(...) which performs the actual data transfer.
155+
1. Command-line arguments are parsed and logging is configured.
156+
2. The SQL Server connection is tested (including permission checks).
157+
3. SharePoint credentials are loaded from the configuration file.
158+
4. Based on the mode (`daily` or `monthly`), `RefreshSQLLists.SPOtoSQLUpdate(...)` performs the data transfer.
159+
5. Data quality routines (`ActivitiesDQ`, `InvoiceRequestDQ`, etc.) run as part of the sync.
160+
6. All actions are logged to the console at the specified verbosity level.
140161

141-
Data quality routines (ActivitiesDQ, InvoiceRequestDQ, etc.) are executed as part of the sync process.
162+
---
142163

143-
All actions are logged to the console according to the verbosity level.
164+
## Project Structure
144165

145-
Project Structure
146-
text
166+
```
147167
ConsoleApp1/
148168
├── ConsoleApp1.csproj
149169
├── packages.config
@@ -166,28 +186,33 @@ ConsoleApp1/
166186
└── XmlConfig/
167187
├── ConfigHelper.cs
168188
└── UserConfig.xml
169-
Key components:
189+
```
170190

171-
RefreshSPOLists – main entry point (Main method), argument handling, SQL test, and orchestration.
191+
| Component | Description |
192+
|---|---|
193+
| `RefreshSPOLists` | Main entry point — argument handling, SQL test, and orchestration |
194+
| `SPOList` / `SPOUser` | Wrappers for SharePoint CSOM operations |
195+
| `ActivitiesDQ` / `InvoiceRequestDQ` | Data quality fixes for specific lists |
196+
| `RefreshSQLLists` | Handles the SharePoint → SQL data transfer |
197+
| `ConfigHelper` | Reads the XML configuration file |
198+
| `Logger` | Console logger with configurable verbosity levels |
172199

173-
SPOList / SPOUser – wrappers for SharePoint CSOM operations.
200+
---
174201

175-
ActivitiesDQ / InvoiceRequestDQ – data quality fixes for specific lists.
202+
## Troubleshooting
176203

177-
RefreshSQLLists – handles the actual SharePoint-to-SQL data transfer.
204+
| Problem | Possible Solution |
205+
|---|---|
206+
| Cannot connect to SQL Server | Check VPN, firewall rules, and the connection string. Run `diagnostic` mode for more details. |
207+
| SharePoint login fails | Verify username/password in the config file. Ensure the account has access to the specified site. |
208+
| "Field not found" errors | Data quality classes expect specific field names (e.g. `OpportunityID`). If your lists differ, modify the source. |
209+
| Throttling / slow performance | The tool batches updates (e.g. 80 items per batch) to avoid SharePoint limits. Adjust batch sizes if needed. |
210+
| Missing tables in SQL | The tool assumes tables already exist with the correct schema. Review `SQLInteraction.cs` for table creation logic. |
178211

179-
ConfigHelper – reads the XML configuration file.
212+
If you encounter unexpected behaviour, run with `--verbose=3` for full debug output and open an issue with the log attached.
180213

181-
Logger – simple console logger with verbosity levels.
214+
---
182215

183-
Troubleshooting
184-
Problem Possible Solution
185-
Cannot connect to SQL Server Check VPN, firewall rules, and the connection string. Run diagnostic mode for more details.
186-
SharePoint login fails Verify username/password in config. Ensure the account has access to the specified site.
187-
"Field not found" errors The data quality classes expect specific field names (e.g., OpportunityID). If your lists use different names, you must modify the code.
188-
Throttling / slow performance The tool batches updates (e.g., 80 items per batch) to avoid SharePoint limits. Adjust batch sizes if needed.
189-
Missing tables in SQL The tool assumes tables already exist with the correct schema. Review SQLInteraction.cs for table creation logic.
190-
If you encounter unexpected behaviour, run with --verbose=3 to get detailed debug output and open an issue with the log.
216+
## License
191217

192-
License
193-
This project is licensed under the MIT License – see the LICENSE file for details.
218+
This project is licensed under the [MIT License](https://opensource.org/license/mit).

0 commit comments

Comments
 (0)