http://techwiki.co.uk/api.php?action=feedcontributions&user=Adam.birds&feedformat=atomTech Wiki - User contributions [en]2024-03-29T14:50:56ZUser contributionsMediaWiki 1.34.0http://techwiki.co.uk/index.php?title=Change_WordPress_Domain&diff=195Change WordPress Domain2023-03-08T19:27:15Z<p>Adam.birds: Created page with "'''To change a WordPress domain execute the following MySQL Commands:''' <pre> UPDATE wp_options SET option_value = replace(option_value, 'https://oldomain.com', 'https://new..."</p>
<hr />
<div>'''To change a WordPress domain execute the following MySQL Commands:'''<br />
<br />
<pre><br />
UPDATE wp_options SET option_value = replace(option_value, 'https://oldomain.com', 'https://newdomain.com') WHERE option_name = 'home' OR option_name = 'siteurl';<br />
</pre><br />
<pre><br />
UPDATE wp_posts SET guid = replace(guid, 'https://oldomain.com', 'https://newdomain.com');<br />
</pre><br />
<pre><br />
UPDATE wp_posts SET post_content = replace(post_content, 'https://oldomain.com', 'https://newdomain.com');<br />
</pre><br />
<pre><br />
UPDATE wp_postmeta SET meta_value = replace(meta_value, 'https://oldomain.com', 'https://newdomain.com');<br />
</pre><br />
<br />
[[Category:Wordpress]]<br />
[[Category:MySQL]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Create_MySQL_User_And_Grant_Privileges&diff=194Create MySQL User And Grant Privileges2023-03-08T19:06:42Z<p>Adam.birds: Created page with "To create a MySQL user and grant privileges run the below commands: <pre> CREATE USER 'username'@'localhost' IDENTIFIED BY 'SuperSecurePassword'; </pre> <pre> GRANT ALL PRIV..."</p>
<hr />
<div>To create a MySQL user and grant privileges run the below commands:<br />
<br />
<pre><br />
CREATE USER 'username'@'localhost' IDENTIFIED BY 'SuperSecurePassword';<br />
</pre><br />
<br />
<pre><br />
GRANT ALL PRIVILEGES ON databasename . * TO 'username'@'localhost';<br />
</pre><br />
<br />
<pre><br />
FLUSH PRIVILEGES;<br />
</pre><br />
<br />
[[Category:MySQL]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Make_script_to_silence_output_unless_errors_and_always_recompile&diff=193Make script to silence output unless errors and always recompile2022-01-29T02:07:41Z<p>Adam.birds: Created page with "'''Here is the script to silence output when running make unless their are errors, it will also always recompile too:''' <pre> #!/bin/bash # Ensure no targets end with .c ar..."</p>
<hr />
<div>'''Here is the script to silence output when running make unless their are errors, it will also always recompile too:'''<br />
<br />
<pre><br />
#!/bin/bash<br />
<br />
# Ensure no targets end with .c<br />
args=""<br />
invalid_args=0<br />
for arg; do<br />
case "$arg" in<br />
(*.c) arg=${arg%.c}; invalid_args=1;;<br />
esac<br />
args="$args $arg"<br />
done<br />
if [ $invalid_args -eq 1 ]; then<br />
echo "Did you mean 'make$args'?"<br />
exit 1<br />
fi<br />
<br />
# Run make<br />
/usr/bin/make -B -s $*<br />
</pre><br />
<br />
'''Copy the above into the below file:'''<br />
<br />
<pre><br />
/usr/local/bin/make<br />
</pre><br />
<br />
Then run:<br />
<br />
<pre><br />
sudo chmod +x /usr/local/bin/make<br />
</pre><br />
<br />
[[Category:Make]]<br />
[[Category:Nginx]]<br />
[[Category:Bash Scripts]]<br />
[[Category:Scripts]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Make&diff=192Category:Make2022-01-29T02:01:50Z<p>Adam.birds: Created page with "This will include everything to do with the Linux command Make! Category:Linux Category:Contents"</p>
<hr />
<div>This will include everything to do with the Linux command Make!<br />
<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=191Main Page2021-10-26T22:33:17Z<p>Adam.birds: </p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text files all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Amazon S3|Amazon S3]]<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Gitlab|Gitlab]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClone|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
The site is created and managed by [https://adbwebdesigns.co.uk ADB Web Designs].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=MediaWiki:Sidebar&diff=190MediaWiki:Sidebar2021-10-22T20:27:12Z<p>Adam.birds: Created page with " * navigation ** mainpage|mainpage-description ** recentchanges-url|recentchanges ** randompage-url|randompage ** helppage|help-mediawiki * Credits for Site ** https://adbwebd..."</p>
<hr />
<div><br />
* navigation<br />
** mainpage|mainpage-description<br />
** recentchanges-url|recentchanges<br />
** randompage-url|randompage<br />
** helppage|help-mediawiki<br />
* Credits for Site<br />
** https://adbwebdesigns.co.uk|ADB Web Designs<br />
** https://adbtechnology.co.uk|ADB Technology<br />
* SEARCH<br />
* TOOLBOX<br />
* LANGUAGES</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=189Main Page2021-10-22T19:51:02Z<p>Adam.birds: Added link to ADB Web Designs</p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text file all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Amazon S3|Amazon S3]]<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Gitlab|Gitlab]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClone|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
The site is created and managed by [https://adbwebdesigns.co.uk ADB Web Designs].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Wordpress_Extra_Theme_Hide_Arrow_in_Drop_Down_Menu&diff=188Wordpress Extra Theme Hide Arrow in Drop Down Menu2019-01-15T22:11:55Z<p>Adam.birds: </p>
<hr />
<div>'''To hide the arrow in the drop-down menu of the Extra theme in Wordpress follow the below:'''<br />
<br />
Login to Wordpress Admin<br />
<br />
Then click Extra and then Theme Options<br />
<br />
Scroll down to the bottom and the enter the below under '''custom.css'''<br />
<br />
<pre><br />
header.header li.menu-item-has-children>a:after {<br />
display: none; <br />
}<br />
<br />
#et-navigation>ul>li.menu-item-has-children>a:before {<br />
width: 100%;<br />
}<br />
</pre><br />
<br />
[[Category:Wordpress]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Wordpress_Extra_Theme_Hide_Arrow_in_Drop_Down_Menu&diff=187Wordpress Extra Theme Hide Arrow in Drop Down Menu2019-01-15T22:11:37Z<p>Adam.birds: Created page with "'''To hide the arrow in the drop-down menu of the Extra theme in Wordpress follow the below:''' Login to Wordpress Admin Then click Extra and then Theme Options Scroll down..."</p>
<hr />
<div>'''To hide the arrow in the drop-down menu of the Extra theme in Wordpress follow the below:'''<br />
<br />
Login to Wordpress Admin<br />
<br />
Then click Extra and then Theme Options<br />
<br />
Scroll down to the bottom and the enter the below under '''custom.css'''<br />
<br />
<br />
<pre><br />
header.header li.menu-item-has-children>a:after {<br />
display: none; <br />
}<br />
<br />
#et-navigation>ul>li.menu-item-has-children>a:before {<br />
width: 100%;<br />
}<br />
</pre><br />
<br />
[[Category:Wordpress]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=The_Trust_Relationship_Between_This_Workstation_and_the_Primary_Domain_Failed&diff=186The Trust Relationship Between This Workstation and the Primary Domain Failed2016-12-09T23:56:51Z<p>Adam.birds: Created page with "When you get the '''The Trust Relationship Between This Workstation and the Primary Domain Failed''' error run the below in '''Powershell''' as '''Administrator''': <pre> Tes..."</p>
<hr />
<div>When you get the '''The Trust Relationship Between This Workstation and the Primary Domain Failed''' error run the below in '''Powershell''' as '''Administrator''':<br />
<br />
<pre><br />
Test-ComputerSecureChannel -Repair -Credential (get-credential)<br />
</pre><br />
<br />
Then enter your '''Domain Admin Credentials'''<br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Reset_Domain_Trust_Relationship&diff=185Reset Domain Trust Relationship2016-11-21T09:50:49Z<p>Adam.birds: </p>
<hr />
<div>If you get '''The trust relationship between this workstation and the primary domain failed''' when attempting to logon then run the following as Admin in Powershell:<br />
<br />
<pre><br />
Test-ComputerSecureChannel -Repair -Credential (get-credential)<br />
</pre><br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Reset_Domain_Trust_Relationship&diff=184Reset Domain Trust Relationship2016-11-21T09:50:39Z<p>Adam.birds: Created page with "If you get '''he trust relationship between this workstation and the primary domain failed''' when attempting to logon then run the following as Admin in Powershell: <pre> Te..."</p>
<hr />
<div>If you get '''he trust relationship between this workstation and the primary domain failed''' when attempting to logon then run the following as Admin in Powershell:<br />
<br />
<pre><br />
Test-ComputerSecureChannel -Repair -Credential (get-credential)<br />
</pre><br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Citrix&diff=183Category:Citrix2016-11-01T22:43:00Z<p>Adam.birds: </p>
<hr />
<div>This will contain all the Citrix stuff.<br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Windows_Updates&diff=182Category:Windows Updates2016-11-01T22:42:31Z<p>Adam.birds: Created page with "Category:Windows Category:Contents"</p>
<hr />
<div>[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Windows_Update_Hangs_on_Checking_for_Updates&diff=181Windows Update Hangs on Checking for Updates2016-11-01T22:42:05Z<p>Adam.birds: </p>
<hr />
<div>We emphasise that for this to work, you must follow exactly as this describes.<br />
<br />
There are two main types of Windows 7 installations: '''32 bit''' or '''64 bit'''. You need to know what is installed in your computer. Click the Start globe, type system in the box. Choose System information in the list. The techie shorthand for 32 bit is x86 and for 64 bit is x64.<br />
<br />
1. Start Windows Update and change the Setting to Never check for updates. Close the Windows Update window. Some members recommend you return the setting to what ever you used before this operation, after you complete Windows Update. IF you do leave the setting at NEVER, Windows Update will no longer be automatic. From this point onwards, you are responsible for starting and installing updates. We recommend you do that a few days following the 2nd Tuesday each month.<br />
<br />
2. Restart your computer.<br />
<br />
3. The process starts by ensuring that there is nothing running that will prevent the update from working properly. Click on the Start globe and type '''services''' into the text box. Click on Services in the list that pops up. In the window that then pops up (its alphabetic), find Windows Update. '''Right-click''' on it (left-click if your mouse is set for left-handers) and choose '''Stop'''. <br />
<br />
4. You are now going to download and install either one or two updates manually. In most cases only the first (KB3172605) of these is needed. If that produces a result that says the update is not appropriate for you computer, you need to first install the 2nd of these (KB3020369), then install the first (KB3172605). Choose the one that is for your machine -- 32 bit (X86) or 64 bit (X64).<br />
<br />
'''KB3172605:'''<br />
<br />
32 bit:<br />
http://download.windowsupdate.com/d/msdownload/update/software/updt/2016/09/windows6.1-kb3172605-x86_ae03ccbd299e434ea2239f1ad86f164e5f4deeda.msu<br />
<br />
64 bit:<br />
http://download.windowsupdate.com/d/msdownload/update/software/updt/2016/09/windows6.1-kb3172605-x64_2bb9bc55f347eee34b1454b50c436eb6fd9301fc.msu<br />
<br />
'''KB3020369:'''<br />
<br />
32 bit:<br />
https://www.microsoft.com/en-us/download/details.aspx?id=46827<br />
<br />
64 bit:<br />
https://www.microsoft.com/en-us/download/details.aspx?id=46817<br />
<br />
5. After restarting your computer, wait about 10 minutes until Windows Update completes its tasks. Do not use the computer for any other purpose during this wait period.<br />
<br />
6. Start Windows Update. It will take only a few minutes (unless, of course it has been many months since the last update) to come up with a list and download the updates you select. The process is quite normal as it always was from this point onward.<br />
<br />
If your computer has not been updated for a long time, it may take longer.<br />
<br />
[[Category:Windows Updates]]<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Windows_Update_Hangs_on_Checking_for_Updates&diff=180Windows Update Hangs on Checking for Updates2016-11-01T22:41:22Z<p>Adam.birds: Created page with "We emphasise that for this to work, you must follow exactly as this describes. There are two main types of Windows 7 installations: '''32 bit''' or '''64 bit'''. You need to..."</p>
<hr />
<div>We emphasise that for this to work, you must follow exactly as this describes.<br />
<br />
There are two main types of Windows 7 installations: '''32 bit''' or '''64 bit'''. You need to know what is installed in your computer. Click the Start globe, type system in the box. Choose System information in the list. The techie shorthand for 32 bit is x86 and for 64 bit is x64.<br />
<br />
1. Start Windows Update and change the Setting to Never check for updates. Close the Windows Update window. Some members recommend you return the setting to what ever you used before this operation, after you complete Windows Update. IF you do leave the setting at NEVER, Windows Update will no longer be automatic. From this point onwards, you are responsible for starting and installing updates. We recommend you do that a few days following the 2nd Tuesday each month.<br />
<br />
2. Restart your computer.<br />
<br />
3. The process starts by ensuring that there is nothing running that will prevent the update from working properly. Click on the Start globe and type '''services''' into the text box. Click on Services in the list that pops up. In the window that then pops up (its alphabetic), find Windows Update. '''Right-click''' on it (left-click if your mouse is set for left-handers) and choose '''Stop'''. <br />
<br />
4. You are now going to download and install either one or two updates manually. In most cases only the first (KB3172605) of these is needed. If that produces a result that says the update is not appropriate for you computer, you need to first install the 2nd of these (KB3020369), then install the first (KB3172605). Choose the one that is for your machine -- 32 bit (X86) or 64 bit (X64).<br />
<br />
'''KB3172605:'''<br />
<br />
32 bit:<br />
http://download.windowsupdate.com/d/msdownload/update/software/updt/2016/09/windows6.1-kb3172605-x86_ae03ccbd299e434ea2239f1ad86f164e5f4deeda.msu<br />
<br />
64 bit:<br />
http://download.windowsupdate.com/d/msdownload/update/software/updt/2016/09/windows6.1-kb3172605-x64_2bb9bc55f347eee34b1454b50c436eb6fd9301fc.msu<br />
<br />
'''KB3020369:'''<br />
<br />
32 bit:<br />
https://www.microsoft.com/en-us/download/details.aspx?id=46827<br />
<br />
64 bit:<br />
https://www.microsoft.com/en-us/download/details.aspx?id=46817<br />
<br />
5. After restarting your computer, wait about 10 minutes until Windows Update completes its tasks. Do not use the computer for any other purpose during this wait period.<br />
<br />
6. Start Windows Update. It will take only a few minutes (unless, of course it has been many months since the last update) to come up with a list and download the updates you select. The process is quite normal as it always was from this point onward.<br />
<br />
If your computer has not been updated for a long time, it may take longer.<br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=WiFi_Profile_Type_Change&diff=179WiFi Profile Type Change2016-11-01T22:26:20Z<p>Adam.birds: Created page with "'''Run the following command as admin in CMD replacing SSID with your wireless name:''' <pre> netsh wlan set profiletype name=SSID profiletype=all </pre> [[Category:Windows]..."</p>
<hr />
<div>'''Run the following command as admin in CMD replacing SSID with your wireless name:'''<br />
<br />
<pre><br />
netsh wlan set profiletype name=SSID profiletype=all<br />
</pre><br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Clear_Out_Old_Kernels&diff=178Clear Out Old Kernels2016-08-30T10:13:21Z<p>Adam.birds: </p>
<hr />
<div>'''To clear out the old kernels from a CentOS box use the following commands:'''<br />
<br />
<pre><br />
yum install yum-utils<br />
package-cleanup --oldkernels --count=2<br />
</pre><br />
<br />
'''To stop the /boot partition from filling again do the following'''<br />
<br />
<pre><br />
add installonly_limit=2 to /etc/yum.conf<br />
</pre><br />
<br />
'''To clear out the old kernels from a Ubuntu box use the following commands:'''<br />
<br />
<pre><br />
apt-get autoremove --purge<br />
</pre><br />
<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Allow_DHCP_through_IPTables&diff=177Allow DHCP through IPTables2016-05-17T09:45:46Z<p>Adam.birds: Created page with "To allow DHCP through IPTbales and IP6Tables run the following commands: <pre> iptables -I INPUT -i eth2 -p udp --dport 67:68 --sport 67:68 -j ACCEPT ip6tables -I INPUT -i et..."</p>
<hr />
<div>To allow DHCP through IPTbales and IP6Tables run the following commands:<br />
<br />
<pre><br />
iptables -I INPUT -i eth2 -p udp --dport 67:68 --sport 67:68 -j ACCEPT<br />
ip6tables -I INPUT -i eth2 -p udp --dport 67:68 --sport 67:68 -j ACCEPT<br />
service iptables save<br />
service ip6tables save<br />
</pre><br />
<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Install_Gitlab_CentOS_7&diff=176Install Gitlab CentOS 72016-05-06T14:08:33Z<p>Adam.birds: Created page with "=Gitlab Installation= ==Prerequisites== *Newly launched CentOS 7 Server *Reasonable Disk Specifications *Internet Connection *Domain Name with DNS pointing to the IP of the..."</p>
<hr />
<div>=Gitlab Installation=<br />
<br />
==Prerequisites==<br />
<br />
*Newly launched CentOS 7 Server<br />
*Reasonable Disk Specifications<br />
*Internet Connection<br />
*Domain Name with DNS pointing to the IP of the server<br />
<br />
==Run Updates==<br />
<br />
It is good practice to update all of the software packages before we proceed. You can do this by running the following command:<br />
<br />
<pre><br />
yum upgrade -y<br />
yum update -y<br />
</pre><br />
<br />
==Set Server Hostname==<br />
<br />
To set the server hostname on CentOS 7 we can use the below commands:<br />
<br />
<pre><br />
hostnamectl set-hostname gitlab.example.com<br />
hostnamectl set-hostname gitlab.example.com --static<br />
</pre><br />
<br />
It is then best practice to reboot the server with the following command:<br />
<br />
<pre><br />
reboot<br />
</pre><br />
<br />
==Install Software Dependencies==<br />
<br />
First of all we need to run the following set of commands to install the correct dependencies:<br />
<br />
<pre><br />
sudo yum install curl policycoreutils openssh-server openssh-clients<br />
sudo systemctl enable sshd<br />
sudo systemctl start sshd<br />
sudo yum install postfix<br />
sudo systemctl enable postfix<br />
sudo systemctl start postfix<br />
</pre><br />
<br />
==Setup Repository==<br />
<br />
To setup the Gitlab Repository run the following command:<br />
<br />
<pre><br />
curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash<br />
</pre><br />
<br />
==Install Package==<br />
<br />
To install the package run the following command:<br />
<br />
<pre><br />
sudo yum install gitlab-ce<br />
</pre><br />
<br />
==Configure Gitlab==<br />
<br />
To run the Gitlab Configuration run the following command:<br />
<br />
<pre><br />
sudo gitlab-ctl reconfigure<br />
</pre><br />
<br />
==Browse to Site and Login For First Time==<br />
<br />
On your first visit, you'll be redirected to a password reset screen to provide the password for the initial administrator account. Enter your desired password and you'll be redirected back to the login screen.<br />
<br />
The default account's username is '''root'''. Provide the password you created earlier and login. After login you can change the username if you wish.<br />
<br />
[[Category:Gitlab]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=175Main Page2016-05-06T11:52:08Z<p>Adam.birds: </p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text file all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Amazon S3|Amazon S3]]<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Gitlab|Gitlab]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClone|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Gitlab&diff=174Category:Gitlab2016-05-06T11:51:30Z<p>Adam.birds: Created page with "This will contain all the stuff about Gitlab! Category:Contents"</p>
<hr />
<div>This will contain all the stuff about Gitlab!<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Microsoft_Office_Group_Policy_Templates&diff=173Microsoft Office Group Policy Templates2016-05-05T21:49:37Z<p>Adam.birds: Created page with "=Importing AMDX/ADML/ADM Templates into Central Store= ==Downloading ADMX/ADML/ADM Templates== First we need to download the appropriate ADMX/ADML/ADM templates for your ver..."</p>
<hr />
<div>=Importing AMDX/ADML/ADM Templates into Central Store=<br />
<br />
==Downloading ADMX/ADML/ADM Templates==<br />
<br />
First we need to download the appropriate ADMX/ADML/ADM templates for your version of Office:<br />
<br />
===Office 2010===<br />
<br />
https://www.microsoft.com/en-us/download/details.aspx?id=18968<br />
<br />
http://download.microsoft.com/download/6/D/1/6D113C3D-4651-4DE3-A501-7B602B0E0DEC/Outlk14-simplemapi.adm<br />
<br />
===Office 2013===<br />
<br />
https://www.microsoft.com/en-us/download/confirmation.aspx?id=35554<br />
<br />
===Office 2016===<br />
<br />
https://www.microsoft.com/en-us/download/confirmation.aspx?id=49030<br />
<br />
==Importing ADMX/ADML/ADM Templates==<br />
<br />
===Office 2010===<br />
<br />
Run the Office 2010 downloaded ADMX/ADML .exe file and saved the extracted files for example to:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2010<br />
</pre><br />
<br />
Go to your SYSVOL Policy Definitions folder for example:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .admx files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2010\ADMX<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .adml files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2010\ADMX\en-us<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions\en-us<br />
</pre><br />
<br />
Copy the .adm file from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2010<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions\en-us<br />
</pre><br />
<br />
===Office 2013===<br />
<br />
Run the Office 2013 downloaded ADMX/ADML .exe file and saved the extracted files for example to:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2013<br />
</pre><br />
<br />
Go to your SYSVOL Policy Definitions folder for example:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .admx files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2013\ADMX<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .adml files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2013\ADMX\en-us<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions\en-us<br />
</pre><br />
<br />
===Office 2016===<br />
<br />
Run the Office 2016 downloaded ADMX/ADML .exe file and saved the extracted files for example to:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2016<br />
</pre><br />
<br />
Go to your SYSVOL Policy Definitions folder for example:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .admx files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2016\ADMX<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions<br />
</pre><br />
<br />
Copy the .adml files from:<br />
<br />
<pre><br />
C:\Users\Administrator\Downloads\Office Templates\2016\ADMX\en-us<br />
</pre><br />
<br />
to:<br />
<br />
<pre><br />
\\example-dc-01\SYSVOL\example.com\Policies\PolicyDefinitions\en-us<br />
</pre><br />
<br />
[[Category:Group Policy]]<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Windows_Server_Roles_Won%27t_Install&diff=172Windows Server Roles Won't Install2016-05-01T20:27:30Z<p>Adam.birds: Created page with "'''If you get an error during installation of Server Roles in Windows Server 2012 R2 the issue is caused by the below two Windows Updates:''' <pre> KB 3098779 KB 3097997 </pr..."</p>
<hr />
<div>'''If you get an error during installation of Server Roles in Windows Server 2012 R2 the issue is caused by the below two Windows Updates:'''<br />
<br />
<pre><br />
KB 3098779<br />
KB 3097997<br />
</pre><br />
<br />
'''Un-installing the above updates and then rebooting the server will resolve the issues.'''<br />
<br />
[[Category:Windows]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=171Main Page2016-05-01T20:04:24Z<p>Adam.birds: </p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text file all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Amazon S3|Amazon S3]]<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClone|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Amazon_S3&diff=170Category:Amazon S32016-05-01T20:00:15Z<p>Adam.birds: Created page with "Category:Contents"</p>
<hr />
<div>[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Find_Size_of_Amazon_S3_Bucket&diff=169Find Size of Amazon S3 Bucket2016-05-01T19:59:42Z<p>Adam.birds: Created page with "'''To find the size of an Amazon S3 Bucket use the following command:''' <pre> aws s3api list-objects --bucket bucketname --output json --query "[sum(Contents[].Size), length..."</p>
<hr />
<div>'''To find the size of an Amazon S3 Bucket use the following command:'''<br />
<br />
<pre><br />
aws s3api list-objects --bucket bucketname --output json --query "[sum(Contents[].Size), length(Contents[])]"<br />
</pre><br />
<br />
This requires you to first run:<br />
<br />
<pre><br />
aws configure<br />
</pre><br />
<br />
To set up your account credentials with the api.<br />
<br />
[[Category:Amazon S3]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=168Main Page2016-04-30T22:27:36Z<p>Adam.birds: </p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text file all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClone|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Main_Page&diff=167Main Page2016-04-30T22:27:09Z<p>Adam.birds: </p>
<hr />
<div>{|style="width: 100%;"<!--CONTENT TABLE START--><br />
|colspan="2"|<br />
<!--WELCOME TABLE START--><br />
{|style="width: 100%; border: 1px solid #ddd; background-color: #f9f9f9"<br />
|style="font-size: 14pt; width: 51%"|Welcome to Tech Wiki<br />
<br />
* Containing [[Special:Statistics|{{NUMBEROFPAGES}}]] pages.<br />
* With [[Special:Statistics|{{NUMBEROFUSERS}}]] editors!<br />
|}<br />
<!--WELCOME TABLE END--><br />
<br />
The purpose of this Wiki is to document and maintain a central repository of fixes/tutorials/information to replace the individual text file all techies keep.<br />
<br />
The below topics are currently documented in this wiki:<br />
<br />
*[[:Category:Backups|Backups]]<br />
*[[:Category:Bacula|Bacula]]<br />
*[[:Category:Bash Scripts|Bash Scripts]]<br />
*[[:Category:Citrix|Citrix]]<br />
*[[:Category:DPM|DPM]]<br />
*[[:Category:Group Policy|Group Policy]]<br />
*[[:Category:Joomla|Joomla]]<br />
*[[:Category:Linux|Linux]]<br />
*[[:Category:Magento|Magento]]<br />
*[[:Category:MediaWiki|MediaWiki]]<br />
*[[:Category:Moodle|Moodle]]<br />
*[[:Category:MySQL|MySQL]]<br />
*[[:Category:Nginx|Nginx]]<br />
*[[:Category:Opencart|Opencart]]<br />
*[[:Category:Openstack|Openstack]]<br />
*[[:Category:PHP|PHP]]<br />
*[[:Category:Prestashop|Prestashop]]<br />
*[[:Category:RClome|RClone]]<br />
*[[:Category:Scripts|Scripting]]<br />
*[[:Category:Windows|Windows]]<br />
*[[:Category:Wordpress|Wordpress]]<br />
<br />
To view the full contents of this Wiki visit the [[:Category:Contents|Contents Page]].<br />
<br />
This page was last updated by [[{{REVISIONUSER}}]].<br />
<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Moodle&diff=166Category:Moodle2016-04-30T21:52:06Z<p>Adam.birds: </p>
<hr />
<div>[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Opencart&diff=165Category:Opencart2016-04-30T21:51:19Z<p>Adam.birds: Created page with "Category:Contents"</p>
<hr />
<div>[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Opencart_Nginx_Config&diff=164Opencart Nginx Config2016-04-30T21:50:54Z<p>Adam.birds: </p>
<hr />
<div>'''Here is the Nginx Config file for Opencart:'''<br />
<br />
<pre><br />
upstream examplecombackend {<br />
server unix:/var/run/php-fcgi-examplecom.sock;<br />
}<br />
<br />
server {<br />
listen 80;<br />
server_name example.com;<br />
root /var/www/vhosts/example.com/htdocs;<br />
index index.php index.html;<br />
<br />
access_log /var/www/vhosts/example.com/access.log;<br />
error_log /var/www/vhosts/example.com/error.log warn;<br />
<br />
location /image/data {<br />
autoindex on;<br />
}<br />
location /admin {<br />
index index.php;<br />
}<br />
location / {<br />
try_files $uri @opencart;<br />
index index.php index.html;<br />
}<br />
location @opencart {<br />
rewrite ^/(.+)$ /index.php?_route_=$1 last;<br />
}<br />
<br />
# Make sure files with the following extensions do not get loaded by nginx because nginx would display the source code, and these files can contain PASSWORDS!<br />
location ~* \.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$|\.php_ {<br />
deny all;<br />
}<br />
# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).<br />
location ~ /\. {<br />
deny all;<br />
access_log off;<br />
log_not_found off;<br />
}<br />
location ~* \.(jpg|jpeg|png|gif|css|js|ico)$ {<br />
expires max;<br />
log_not_found off;<br />
}<br />
location ~ \.php$ {<br />
root /var/www/vhosts/example.com/htdocs;<br />
try_files $uri =404;<br />
fastcgi_pass examplecombackend;<br />
fastcgi_index index.php;<br />
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br />
include fastcgi_params;<br />
}<br />
}<br />
<br />
</pre><br />
<br />
[[Category:Opencart]]<br />
[[Category:Nginx]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Opencart_Nginx_Config&diff=163Opencart Nginx Config2016-04-30T21:50:26Z<p>Adam.birds: </p>
<hr />
<div>'''Here is the Nginx Config file for Opencart:'''<br />
<br />
<pre><br />
upstream examplecombackend {<br />
server unix:/var/run/php-fcgi-examplecom.sock;<br />
}<br />
<br />
server {<br />
listen 80;<br />
server_name example.com;<br />
root /var/www/vhosts/example.com/htdocs;<br />
index index.php index.html;<br />
<br />
access_log /var/www/vhosts/example.com/access.log;<br />
error_log /var/www/vhosts/example.com/error.log warn;<br />
<br />
location /image/data {<br />
autoindex on;<br />
}<br />
location /admin {<br />
index index.php;<br />
}<br />
location / {<br />
try_files $uri @opencart;<br />
index index.php index.html;<br />
}<br />
location @opencart {<br />
rewrite ^/(.+)$ /index.php?_route_=$1 last;<br />
}<br />
<br />
# Make sure files with the following extensions do not get loaded by nginx because nginx would display the source code, and these files can contain PASSWORDS!<br />
location ~* \.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$|\.php_ {<br />
deny all;<br />
}<br />
# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).<br />
location ~ /\. {<br />
deny all;<br />
access_log off;<br />
log_not_found off;<br />
}<br />
location ~* \.(jpg|jpeg|png|gif|css|js|ico)$ {<br />
expires max;<br />
log_not_found off;<br />
}<br />
location ~ \.php$ {<br />
root /var/www/vhosts/example.com/htdocs;<br />
try_files $uri =404;<br />
fastcgi_pass examplecombackend;<br />
fastcgi_index index.php;<br />
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br />
include fastcgi_params;<br />
}<br />
}<br />
<br />
</pre><br />
<br />
[[Categroy:Opencart]]<br />
[[Category:Nginx]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Filtering&diff=162RClone - Filtering2016-04-30T21:40:15Z<p>Adam.birds: </p>
<hr />
<div>=Filtering, includes and excludes=<br />
<br />
Rclone has a sophisticated set of include and exclude rules. Some of these are based on patterns and some on other things like file size.<br />
<br />
The filters are applied for the '''copy''', '''sync''', '''move''', '''ls''', '''lsl''', '''md5sum''', '''sha1sum''', '''size''', '''delete''' and '''check''' operations. Note that '''purge''' does not obey the filters.<br />
<br />
Each path as it passes through rclone is matched against the include and exclude rules like '''--include''', '''--exclude''', '''--include-from''', '''--exclude-from''', '''--filter''', or '''--filter-from'''. The simplest way to try them out is using the '''ls''' command, or '''--dry-run''' together with '''-v'''.<br />
<br />
==Patterns==<br />
<br />
The patterns used to match files for inclusion or exclusion are based on “file globs” as used by the unix shell.<br />
<br />
If the pattern starts with a '''/''' then it only matches at the top level of the directory tree, relative to the root of the remote. If it doesn’t start with '''/''' then it is matched starting at the '''end of the path''', but it will only match a complete path element:<br />
<br />
<pre><br />
file.jpg - matches "file.jpg"<br />
- matches "directory/file.jpg"<br />
- doesn't match "afile.jpg"<br />
- doesn't match "directory/afile.jpg"<br />
/file.jpg - matches "file.jpg" in the root directory of the remote<br />
- doesn't match "afile.jpg"<br />
- doesn't match "directory/file.jpg"<br />
</pre><br />
<br />
'''Important''' Note that you must use '''/''' in patterns and not '''\''' even if running on Windows.<br />
<br />
A '''*''' matches anything but not a '''/'''.<br />
<br />
<pre><br />
*.jpg - matches "file.jpg"<br />
- matches "directory/file.jpg"<br />
- doesn't match "file.jpg/something"<br />
</pre><br />
<br />
Use '''**''' to match anything, including slashes ('''/''').<br />
<br />
<pre><br />
dir/** - matches "dir/file.jpg"<br />
- matches "dir/dir1/dir2/file.jpg"<br />
- doesn't match "directory/file.jpg"<br />
- doesn't match "adir/file.jpg"<br />
</pre><br />
<br />
A '''?''' matches any character except a slash '''/'''.<br />
<br />
<pre><br />
l?ss - matches "less"<br />
- matches "lass"<br />
- doesn't match "floss"<br />
</pre><br />
<br />
A '''[''' and ''']''' together make a a character class, such as '''[a-z]''' or '''[aeiou]''' or '''[[:alpha:]]'''. See the <span class="plainlinks">[https://golang.org/pkg/regexp/syntax/ go regexp docs]</span> for more info on these.<br />
<br />
<pre><br />
h[ae]llo - matches "hello"<br />
- matches "hallo"<br />
- doesn't match "hullo"<br />
</pre><br />
<br />
A '''{''' and '''}''' define a choice between elements. It should contain a comma seperated list of patterns, any of which might match. These patterns can contain wildcards.<br />
<br />
<pre><br />
{one,two}_potato - matches "one_potato"<br />
- matches "two_potato"<br />
- doesn't match "three_potato"<br />
- doesn't match "_potato"<br />
</pre><br />
<br />
Special characters can be escaped with a '''\''' before them.<br />
<br />
<pre><br />
\*.jpg - matches "*.jpg"<br />
\\.jpg - matches "\.jpg"<br />
\[one\].jpg - matches "[one].jpg"<br />
</pre><br />
<br />
==Differences between rsync and rclone patterns==<br />
<br />
Rclone implements bash style '''{a,b,c}''' glob matching which rsync doesn’t.<br />
<br />
Rclone ignores '''/''' at the end of a pattern.<br />
<br />
Rclone always does a wildcard match so '''\''' must always escape a '''\'''.<br />
<br />
==How the rules are used==<br />
<br />
Rclone maintains a list of include rules and exclude rules.<br />
<br />
Each file is matched in order against the list until it finds a match. The file is then included or excluded according to the rule type.<br />
<br />
If the matcher falls off the bottom of the list then the path is included.<br />
<br />
For example given the following rules, '''+''' being include, '''-''' being exclude,<br />
<br />
<pre><br />
- secret*.jpg<br />
+ *.jpg<br />
+ *.png<br />
+ file2.avi<br />
- *<br />
</pre><br />
<br />
This would include:<br />
<br />
*'''file1.jpg'''<br />
*'''file3.png'''<br />
*'''file2.avi'''<br />
<br />
This would exclude:<br />
<br />
*'''secret17.jpg'''<br />
*non '''*.jpg''' and '''*.png'''<br />
<br />
==Adding filtering rules==<br />
<br />
Filtering rules are added with the following command line flags:<br />
<br />
===--exclude - Exclude files matching pattern===<br />
<br />
Add a single exclude rule with '''--exclude'''.<br />
<br />
Eg '''--exclude *.bak''' to exclude all bak files from the sync.<br />
<br />
===--exclude-from - Read exclude patterns from file===<br />
<br />
Add exclude rules from a file.<br />
<br />
Prepare a file like this '''exclude-file.txt'''<br />
<br />
<pre><br />
# a sample exclude rule file<br />
*.bak<br />
file2.jpg<br />
</pre><br />
<br />
Then use as '''--exclude-from exclude-file.txt'''. This will sync all files except those ending in '''bak''' and '''file2.jpg'''.<br />
<br />
This is useful if you have a lot of rules.<br />
<br />
===--include - Include files matching pattern===<br />
<br />
Add a single include rule with '''--include'''.<br />
<br />
Eg '''--include *.{png,jpg}''' to include all '''png''' and '''jpg''' files in the backup and no others.<br />
<br />
This adds an implicit '''--exclude *''' at the very end of the filter list. This means you can mix '''--include''' and '''--include-from''' with the other filters (eg '''--exclude''') but you must include all the files you want in the include statement. If this doesn’t provide enough flexibility then you must use '''--filter-from'''.<br />
<br />
===--include-from - Read include patterns from file===<br />
<br />
Add include rules from a file.<br />
<br />
Prepare a file like this '''include-file.txt'''<br />
<br />
<pre><br />
# a sample include rule file<br />
*.jpg<br />
*.png<br />
file2.avi<br />
</pre><br />
<br />
Then use as '''--include-from include-file.txt'''. This will sync all '''jpg''', '''png''' files and '''file2.avi'''.<br />
<br />
This is useful if you have a lot of rules.<br />
<br />
This adds an implicit '''--exclude *''' at the very end of the filter list. This means you can mix '''--include''' and '''--include-from''' with the other filters (eg '''--exclude''') but you must include all the files you want in the include statement. If this doesn’t provide enough flexibility then you must use '''--filter-from'''.<br />
<br />
===--filter - Add a file-filtering rule===<br />
<br />
This can be used to add a single include or exclude rule. Include rules start with '''+''' and exclude rules start with '''-'''. A special rule called '''!''' can be used to clear the existing rules.<br />
<br />
Eg '''--filter "- *.bak"''' to exclude all bak files from the sync.<br />
<br />
===--filter-from - Read filtering patterns from a file===<br />
<br />
Add include/exclude rules from a file.<br />
<br />
Prepare a file like this '''filter-file.txt'''<br />
<br />
<pre><br />
# a sample exclude rule file<br />
- secret*.jpg<br />
+ *.jpg<br />
+ *.png<br />
+ file2.avi<br />
# exclude everything else<br />
- *<br />
</pre><br />
<br />
Then use as '''--filter-from filter-file.txt'''. The rules are processed in the order that they are defined.<br />
<br />
This example will include all '''jpg''' and '''png''' files, exclude any files matching '''secret*.jpg''' and include '''file2.avi'''. Everything else will be excluded from the sync.<br />
<br />
===--files-from - Read list of source-file names===<br />
<br />
This reads a list of file names from the file passed in and '''only''' these files are transferred. The filtering rules are ignored completely if you use this option.<br />
<br />
Prepare a file like this '''files-from.txt'''<br />
<br />
<pre><br />
# comment<br />
file1.jpg<br />
file2.jpg<br />
</pre><br />
<br />
Then use as '''--files-from files-from.txt'''. This will only transfer '''file1.jpg''' and '''file2.jpg''' providing they exist.<br />
<br />
===--min-size - Don’t transfer any file smaller than this===<br />
<br />
This option controls the minimum size file which will be transferred. This defaults to '''kBytes''' but a suffix of '''k''', '''M''', or '''G''' can be used.<br />
<br />
For example '''--min-size 50k''' means no files smaller than 50kByte will be transferred.<br />
<br />
===--max-size - Don’t transfer any file larger than this===<br />
<br />
This option controls the maximum size file which will be transferred. This defaults to '''kBytes''' but a suffix of '''k''', '''M''', or '''G''' can be used.<br />
<br />
For example '''--max-size 1G''' means no files larger than 1GByte will be transferred.<br />
<br />
===--max-age - Don’t transfer any file older than this===<br />
<br />
This option controls the maximum age of files to transfer. Give in seconds or with a suffix of:<br />
<br />
*'''ms''' - Milliseconds<br />
*'''s''' - Seconds<br />
*'''m''' - Minutes<br />
*'''h''' - Hours<br />
*'''d''' - Days<br />
*'''w''' - Weeks<br />
*'''M''' - Months<br />
*'''y''' - Years<br />
<br />
For example '''--max-age 2d''' means no files older than 2 days will be transferred.<br />
<br />
===--min-age - Don’t transfer any file younger than this===<br />
<br />
This option controls the minimum age of files to transfer. Give in seconds or with a suffix (see '''--max-age''' for list of suffixes)<br />
<br />
For example '''--min-age 2d''' means no files younger than 2 days will be transferred.<br />
<br />
===--delete-excluded - Delete files on dest excluded from sync===<br />
<br />
'''Important''' this flag is dangerous - use with '''--dry-run''' and '''-v''' first.<br />
<br />
When doing '''rclone sync''' this will delete any files which are excluded from the sync on the destination.<br />
<br />
If for example you did a sync from '''A''' to '''B''' without the '''--min-size 50k''' flag<br />
<br />
<pre><br />
rclone sync A: B:<br />
</pre><br />
<br />
Then you repeated it like this with the '''--delete-excluded'''<br />
<br />
<pre><br />
rclone --min-size 50k --delete-excluded sync A: B:<br />
</pre><br />
<br />
This would delete all files on '''B''' which are less than 50 kBytes as these are now excluded from the sync.<br />
<br />
Always test first with '''--dry-run''' and '''-v''' before using this flag.<br />
<br />
===--dump-filters - dump the filters to the output===<br />
<br />
This dumps the defined filters to the output as regular expressions.<br />
<br />
Useful for debugging.<br />
<br />
==Quoting shell metacharacters==<br />
<br />
The examples above may not work verbatim in your shell as they have shell metacharacters in them (eg '''*'''), and may require quoting.<br />
<br />
Eg linux, OSX<br />
<br />
*'''--include \*.jpg'''<br />
*'''--include '*.jpg''''<br />
*'''--include='*.jpg''''<br />
<br />
In Windows the expansion is done by the command not the shell so this should work fine<br />
<br />
*'''--include *.jpg'''<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Filtering&diff=161RClone - Filtering2016-04-30T21:00:53Z<p>Adam.birds: Created page with "=Filtering, includes and excludes= Rclone has a sophisticated set of include and exclude rules. Some of these are based on patterns and some on other things like file size...."</p>
<hr />
<div>=Filtering, includes and excludes=<br />
<br />
Rclone has a sophisticated set of include and exclude rules. Some of these are based on patterns and some on other things like file size.<br />
<br />
The filters are applied for the '''copy''', '''sync''', '''move''', '''ls''', '''lsl''', '''md5sum''', '''sha1sum''', '''size''', '''delete''' and '''check''' operations. Note that '''purge''' does not obey the filters.<br />
<br />
Each path as it passes through rclone is matched against the include and exclude rules like '''--include''', '''--exclude''', '''--include-from''', '''--exclude-from''', '''--filter''', or '''--filter-from'''. The simplest way to try them out is using the '''ls''' command, or '''--dry-run''' together with '''-v'''.<br />
<br />
==Patterns==<br />
<br />
The patterns used to match files for inclusion or exclusion are based on “file globs” as used by the unix shell.<br />
<br />
If the pattern starts with a '''/''' then it only matches at the top level of the directory tree, relative to the root of the remote. If it doesn’t start with '''/''' then it is matched starting at the '''end of the path''', but it will only match a complete path element:<br />
<br />
<pre><br />
file.jpg - matches "file.jpg"<br />
- matches "directory/file.jpg"<br />
- doesn't match "afile.jpg"<br />
- doesn't match "directory/afile.jpg"<br />
/file.jpg - matches "file.jpg" in the root directory of the remote<br />
- doesn't match "afile.jpg"<br />
- doesn't match "directory/file.jpg"<br />
</pre><br />
<br />
'''Important''' Note that you must use '''/''' in patterns and not '''\''' even if running on Windows.</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Usage&diff=160RClone - Usage2016-04-30T20:52:16Z<p>Adam.birds: Created page with "=Usage= RClone syncs a directory tree from one storage system to another. Its syntax is like this: <pre> Syntax: [options] subcommand <parameters> <parameters...> </pre> S..."</p>
<hr />
<div>=Usage=<br />
<br />
RClone syncs a directory tree from one storage system to another.<br />
<br />
Its syntax is like this:<br />
<br />
<pre><br />
Syntax: [options] subcommand <parameters> <parameters...><br />
</pre><br />
<br />
Source and destination paths are specified by the name you gave the storage system in the config file then the sub path, eg “drive:myfolder” to look at “myfolder” in Google drive.<br />
<br />
You can define as many storage paths as you like in the config file.<br />
<br />
==Sub-Commands==<br />
<br />
===rclone copy source:path dest:path===<br />
<br />
Copy the source to the destination. Doesn’t transfer unchanged files, testing by size and modification time or MD5SUM. Doesn’t delete files from the destination.<br />
<br />
Note that it is always the contents of the directory that is synced, not the directory so when source:path is a directory, it’s the contents of source:path that are copied, not the directory name and contents.<br />
<br />
If dest:path doesn’t exist, it is created and the source:path contents go there.<br />
<br />
For example:<br />
<br />
<pre><br />
rclone copy source:sourcepath dest:destpath<br />
</pre><br />
<br />
Let’s say there are two files in sourcepath:<br />
<br />
<pre><br />
sourcepath/one.txt<br />
sourcepath/two.txt<br />
</pre><br />
<br />
This copies them to:<br />
<br />
<pre><br />
destpath/one.txt<br />
destpath/two.txt<br />
</pre><br />
<br />
Not to:<br />
<br />
<pre><br />
destpath/sourcepath/one.txt<br />
destpath/sourcepath/two.txt<br />
</pre><br />
<br />
If you are familiar with '''rsync''', rclone always works as if you had written a trailing / - meaning “copy the contents of this directory”. This applies to all commands and whether you are talking about the source or destination.<br />
<br />
===rclone sync source:path dest:path===<br />
<br />
Sync the source to the destination, changing the destination only. Doesn’t transfer unchanged files, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary.<br />
<br />
'''Important:''' Since this can cause data loss, test first with the '''--dry-run''' flag to see exactly what would be copied and deleted.<br />
<br />
Note that files in the destination won’t be deleted if there were any errors at any point.<br />
<br />
It is always the contents of the directory that is synced, not the directory so when source:path is a directory, it’s the contents of source:path that are copied, not the directory name and contents. See extended explanation in the '''copy''' command above if unsure.<br />
<br />
If dest:path doesn’t exist, it is created and the source:path contents go there.<br />
<br />
===move source:path dest:path===<br />
<br />
Moves the source to the destination.<br />
<br />
If there are no filters in use this is equivalent to a copy followed by a purge, but may using server side operations to speed it up if possible.<br />
<br />
If filters are in use then it is equivalent to a copy followed by delete, followed by an rmdir (which only removes the directory if empty). The individual file moves will be moved with server side operations if possible.<br />
<br />
'''Important''': Since this can cause data loss, test first with the –dry-run flag.<br />
<br />
===rclone ls remote:path===<br />
<br />
List all the objects in the the path with size and path.<br />
<br />
===rclone lsd remote:path===<br />
<br />
List all directories/containers/buckets in the the path.<br />
<br />
===rclone lsl remote:path===<br />
<br />
List all the objects in the the path with modification time, size and path.<br />
<br />
===rclone md5sum remote:path===<br />
<br />
Produces an md5sum file for all the objects in the path. This is in the same format as the standard md5sum tool produces.<br />
<br />
===rclone sha1sum remote:path===<br />
<br />
Produces an sha1sum file for all the objects in the path. This is in the same format as the standard sha1sum tool produces.<br />
<br />
===rclone size remote:path===<br />
<br />
Prints the total size of objects in remote:path and the number of objects.<br />
<br />
===rclone mkdir remote:path===<br />
<br />
Make the path if it doesn’t already exist<br />
<br />
===rclone rmdir remote:path===<br />
<br />
Remove the path. Note that you can’t remove a path with objects in it, use purge for that.<br />
<br />
===rclone purge remote:path===<br />
<br />
Remove the path and all of its contents. Note that this does not obey include/exclude filters - everything will be removed. Use '''delete''' if you want to selectively delete files.<br />
<br />
===rclone delete remote:path===<br />
<br />
Remove the contents of path. Unlike '''purge''' it obeys include/exclude filters so can be used to selectively delete files.<br />
<br />
Eg delete all files bigger than 100MBytes<br />
<br />
Check what would be deleted first (use either)<br />
<br />
<pre><br />
rclone --min-size 100M lsl remote:path<br />
rclone --dry-run --min-size 100M delete remote:path<br />
</pre><br />
<br />
Then delete:<br />
<br />
<pre><br />
rclone --min-size 100M delete remote:path<br />
</pre><br />
<br />
That reads “delete everything with a minimum size of 100 MB”, hence delete all files bigger than 100MBytes.<br />
<br />
===rclone check source:path dest:path===<br />
<br />
Checks the files in the source and destination match. It compares sizes and MD5SUMs and prints a report of files which don’t match. It doesn’t alter the source or destination.<br />
<br />
'''--size-only''' may be used to only compare the sizes, not the MD5SUMs.<br />
<br />
===rclone dedupe remote:path===<br />
<br />
By default '''dedupe''' interactively finds duplicate files and offers to delete all but one or rename them to be different. Only useful with Google Drive which can have duplicate file names.<br />
<br />
The '''dedupe''' command will delete all but one of any identical (same md5sum) files it finds without confirmation. This means that for most duplicated files the '''dedupe''' command will not be interactive. You can use '''--dry-run''' to see what would happen without doing anything.<br />
<br />
Here is an example run.<br />
<br />
Before - with duplicates<br />
<br />
<pre><br />
$ rclone lsl drive:dupes<br />
6048320 2016-03-05 16:23:16.798000000 one.txt<br />
6048320 2016-03-05 16:23:11.775000000 one.txt<br />
564374 2016-03-05 16:23:06.731000000 one.txt<br />
6048320 2016-03-05 16:18:26.092000000 one.txt<br />
6048320 2016-03-05 16:22:46.185000000 two.txt<br />
1744073 2016-03-05 16:22:38.104000000 two.txt<br />
564374 2016-03-05 16:22:52.118000000 two.txt<br />
</pre><br />
<br />
Now the '''dedupe''' session:<br />
<br />
<pre><br />
$ rclone dedupe drive:dupes<br />
2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.<br />
one.txt: Found 4 duplicates - deleting identical copies<br />
one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36")<br />
one.txt: 2 duplicates remain<br />
1: 6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36<br />
2: 564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81<br />
s) Skip and do nothing<br />
k) Keep just one (choose which in next step)<br />
r) Rename all to be different (by changing file.jpg to file-1.jpg)<br />
s/k/r> k<br />
Enter the number of the file to keep> 1<br />
one.txt: Deleted 1 extra copies<br />
two.txt: Found 3 duplicates - deleting identical copies<br />
two.txt: 3 duplicates remain<br />
1: 564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81<br />
2: 6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36<br />
3: 1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802<br />
s) Skip and do nothing<br />
k) Keep just one (choose which in next step)<br />
r) Rename all to be different (by changing file.jpg to file-1.jpg)<br />
s/k/r> r<br />
two-1.txt: renamed from: two.txt<br />
two-2.txt: renamed from: two.txt<br />
two-3.txt: renamed from: two.txt<br />
</pre><br />
<br />
The result being:<br />
<br />
<pre><br />
$ rclone lsl drive:dupes<br />
6048320 2016-03-05 16:23:16.798000000 one.txt<br />
564374 2016-03-05 16:22:52.118000000 two-1.txt<br />
6048320 2016-03-05 16:22:46.185000000 two-2.txt<br />
1744073 2016-03-05 16:22:38.104000000 two-3.txt<br />
</pre><br />
<br />
Dedupe can be run non interactively using the '''--dedupe-mode''' flag.<br />
<br />
* '''--dedupe-mode interactive''' - interactive as above.<br />
* '''--dedupe-mode skip''' - removes identical files then skips anything left.<br />
* '''--dedupe-mode first''' - removes identical files then keeps the first one.<br />
* '''--dedupe-mode newest''' - removes identical files then keeps the newest one.<br />
* '''--dedupe-mode oldest''' - removes identical files then keeps the oldest one.<br />
* '''--dedupe-mode rename''' - removes identical files then renames the rest to be different.<br />
<br />
For example to rename all the identically named photos in your Google Photos directory, do:<br />
<br />
<pre><br />
rclone dedupe --dedupe-mode rename "drive:Google Photos"<br />
</pre><br />
<br />
===rclone config===<br />
<br />
Enter an interactive configuration session.<br />
<br />
===rclone help===<br />
<br />
Prints help on rclone commands and options.<br />
<br />
==Server Side Copy==<br />
<br />
Drive, S3, Dropbox, Swift and Google Cloud Storage support server side copy.<br />
<br />
This means if you want to copy one folder to another then rclone won’t download all the files and re-upload them; it will instruct the server to copy them in place.<br />
<br />
Eg<br />
<br />
<pre><br />
rclone copy s3:oldbucket s3:newbucket<br />
</pre><br />
<br />
Will copy the contents of '''oldbucket''' to '''newbucket''' without downloading and re-uploading.<br />
<br />
Remotes which don’t support server side copy (eg local) '''will''' download and re-upload in this case.<br />
<br />
Server side copies are used with '''sync''' and '''copy''' and will be identified in the log when using the -v flag.<br />
<br />
Server side copies will only be attempted if the remote names are the same.<br />
<br />
This can be used when scripting to make aged backups efficiently, eg<br />
<br />
<pre><br />
rclone sync remote:current-backup remote:previous-backup<br />
rclone sync /path/to/files remote:current-backup<br />
</pre><br />
<br />
==Options==<br />
<br />
Rclone has a number of options to control its behaviour.<br />
<br />
Options which use TIME use the go time parser. A duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as “300ms”, “-1.5h” or “2h45m”. Valid time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”.<br />
<br />
Options which use SIZE use kByte by default. However a suffix of '''k''' for kBytes, '''M''' for MBytes and '''G''' for GBytes may be used. These are the binary units, eg 2**10, 2**20, 2**30 respectively.<br />
<br />
===–bwlimit=SIZE===<br />
<br />
Bandwidth limit in kBytes/s, or use suffix k|M|G. The default is '''0''' which means to not limit bandwidth.<br />
<br />
For example to limit bandwidth usage to 10 MBytes/s use '''--bwlimit 10M'''<br />
<br />
This only limits the bandwidth of the data transfer, it doesn’t limit the bandwith of the directory listings etc.<br />
<br />
===–checkers=N===<br />
<br />
The number of checkers to run in parallel. Checkers do the equality checking of files during a sync. For some storage systems (eg s3, swift, dropbox) this can take a significant amount of time so they are run in parallel.<br />
<br />
The default is to run 8 checkers in parallel.<br />
<br />
===-c, –checksum===<br />
<br />
Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check the file hash and size to determine if files are equal.<br />
<br />
This is useful when the remote doesn’t support setting modified time and a more accurate sync is desired than just checking the file size.<br />
<br />
This is very useful when transferring between remotes which store the same hash type on the object, eg Drive and Swift. For details of which remotes support which hash type see the table in the [[RClone -Overview|overview section]].<br />
<br />
Eg '''rclone --checksum sync s3:/bucket swift:/bucket''' would run much quicker than without the '''--checksum''' flag.<br />
<br />
When using this flag, rclone won’t update mtimes of remote files if they are incorrect as it would normally.<br />
<br />
===–config=CONFIG_FILE===<br />
<br />
Specify the location of the rclone config file. Normally this is in your home directory as a file called '''.rclone.conf'''. If you run '''rclone -h''' and look at the help for the '''--config''' option you will see where the default location is for you. Use this flag to override the config location, eg '''rclone --config=".myconfig" .config.'''<br />
<br />
===–contimeout=TIME===<br />
<br />
Set the connection timeout. This should be in go time format which looks like '''5s''' for 5 seconds, '''10m''' for 10 minutes, or '''3h30m'''.<br />
<br />
The connection timeout is the amount of time rclone will wait for a connection to go through to a remote object storage system. It is '''1m''' by default.<br />
<br />
===–dedupe-mode MODE===<br />
<br />
Mode to run dedupe command in. One of '''interactive''', '''skip''', '''first''', '''newest''', '''oldest''', '''rename'''. The default is '''interactive'''. See the dedupe command for more information as to what these options mean.<br />
<br />
===-n, –dry-run===<br />
<br />
Do a trial run with no permanent changes. Use this to see what rclone would do without actually doing it. Useful when setting up the '''sync''' command which deletes files in the destination.<br />
<br />
===–ignore-existing===<br />
<br />
Using this option will make rclone unconditionally skip all files that exist on the destination, no matter the content of these files.<br />
<br />
While this isn’t a generally recommended option, it can be useful in cases where your files change due to encryption. However, it cannot correct partial transfers in case a transfer was interrupted.<br />
<br />
===-I, –ignore-times===<br />
<br />
Using this option will cause rclone to unconditionally upload all files regardless of the state of files on the destination.<br />
<br />
Normally rclone would skip any files that have the same modification time and are the same size (or have the same checksum if using '''--checksum''').<br />
<br />
===–log-file=FILE===<br />
<br />
Log all of rclone’s output to FILE. This is not active by default. This can be useful for tracking down problems with syncs in combination with the '''-v''' flag.<br />
<br />
===–low-level-retries NUMBER===<br />
<br />
This controls the number of low level retries rclone does.<br />
<br />
A low level retry is used to retry a failing operation - typically one HTTP request. This might be uploading a chunk of a big file for example. You will see low level retries in the log with the '''-v''' flag.<br />
<br />
This shouldn’t need to be changed from the default in normal operations, however if you get a lot of low level retries you may wish to reduce the value so rclone moves on to a high level retry (see the '''--retries''' flag) quicker.<br />
<br />
Disable low level retries with '''--low-level-retries''' 1.<br />
<br />
===–modify-window=TIME===<br />
<br />
When checking whether a file has been modified, this is the maximum allowed time difference that a file can have and still be considered equivalent.<br />
<br />
The default is '''1ns''' unless this is overridden by a remote. For example OS X only stores modification times to the nearest second so if you are reading and writing to an OS X filing system this will be '''1s''' by default.<br />
<br />
This command line flag allows you to override that computed default.<br />
<br />
===–no-gzip-encoding===<br />
<br />
Don’t set '''Accept-Encoding: gzip'''. This means that rclone won’t ask the server for compressed files automatically. Useful if you’ve set the server to return files with '''Content-Encoding: gzip''' but you uploaded compressed files.<br />
<br />
There is no need to set this in normal operation, and doing so will decrease the network transfer efficiency of rclone.<br />
<br />
===-q, –quiet===<br />
<br />
Normally rclone outputs stats and a completion message. If you set this flag it will make as little output as possible.<br />
<br />
===–retries int===<br />
<br />
Retry the entire sync if it fails this many times it fails (default 3).<br />
<br />
Some remotes can be unreliable and a few retries helps pick up the files which didn’t get transferred because of errors.<br />
<br />
Disable retries with '''--retries 1'''.<br />
<br />
===–size-only===<br />
<br />
Normally rclone will look at modification time and size of files to see if they are equal. If you set this flag then rclone will check only the size.<br />
<br />
This can be useful transferring files from dropbox which have been modified by the desktop sync client which doesn’t set checksums of modification times in the same way as rclone.<br />
<br />
When using this flag, rclone won’t update mtimes of remote files if they are incorrect as it would normally.<br />
<br />
===–stats=TIME===<br />
<br />
Rclone will print stats at regular intervals to show its progress.<br />
<br />
This sets the interval.<br />
<br />
The default is '''1m'''. Use 0 to disable.<br />
<br />
===–delete-(before,during,after)===<br />
<br />
This option allows you to specify when files on your destination are deleted when you sync folders.<br />
<br />
Specifying the value '''--delete-before''' will delete all files present on the destination, but not on the source before starting the transfer of any new or updated files.<br />
<br />
Specifying '''--delete-during''' (default value) will delete files while checking and uploading files. This is usually the fastest option.<br />
<br />
Specifying '''--delete-after''' will delay deletion of files until all new/updated files have been successfully transferred.<br />
<br />
===–timeout=TIME===<br />
<br />
This sets the IO idle timeout. If a transfer has started but then becomes idle for this long it is considered broken and disconnected.<br />
<br />
The default is '''5m'''. Set to 0 to disable.<br />
<br />
===–transfers=N===<br />
<br />
The number of file transfers to run in parallel. It can sometimes be useful to set this to a smaller number if the remote is giving a lot of timeouts or bigger if you have lots of bandwidth and a fast remote.<br />
<br />
The default is to run 4 file transfers in parallel.<br />
<br />
===-u, –update===<br />
<br />
This forces rclone to skip any files which exist on the destination and have a modified time that is newer than the source file.<br />
<br />
If an existing destination file has a modification time equal (within the computed modify window precision) to the source file’s, it will be updated if the sizes are different.<br />
<br />
On remotes which don’t support mod time directly the time checked will be the uploaded time. This means that if uploading to one of these remoes, rclone will skip any files which exist on the destination and have an uploaded time that is newer than the modification time of the source file.<br />
<br />
This can be useful when transferring to a remote which doesn’t support mod times directly as it is more accurate than a '''--size-only''' check and faster than using '''--checksum'''.<br />
<br />
===-v, –verbose===<br />
<br />
If you set this flag, rclone will become very verbose telling you about every file it considers and transfers.<br />
<br />
Very useful for debugging.<br />
<br />
===-V, –version===<br />
<br />
Prints the version number<br />
<br />
==Configuration Encryption==<br />
<br />
Your configuration file contains information for logging in to your cloud services. This means that you should keep your '''.rclone.conf''' file in a secure location.<br />
<br />
If you are in an environment where that isn’t possible, you can add a password to your configuration. This means that you will have to enter the password every time you start rclone.<br />
<br />
To add a password to your rclone configuration, execute '''rclone config'''.<br />
<br />
<pre><br />
>rclone config<br />
Current remotes:<br />
<br />
e) Edit existing remote<br />
n) New remote<br />
d) Delete remote<br />
s) Set configuration password<br />
q) Quit config<br />
e/n/d/s/q><br />
</pre><br />
<br />
Go into '''s''', Set configuration password:<br />
<br />
<pre><br />
e/n/d/s/q> s<br />
Your configuration is not encrypted.<br />
If you add a password, you will protect your login information to cloud services.<br />
a) Add Password<br />
q) Quit to main menu<br />
a/q> a<br />
Enter NEW configuration password:<br />
password><br />
Confirm NEW password:<br />
password><br />
Password set<br />
Your configuration is encrypted.<br />
c) Change Password<br />
u) Unencrypt configuration<br />
q) Quit to main menu<br />
c/u/q><br />
</pre><br />
<br />
Your configuration is now encrypted, and every time you start rclone you will now be asked for the password. In the same menu you can change the password or completely remove encryption from your configuration.<br />
<br />
There is no way to recover the configuration if you lose your password.<br />
<br />
rclone uses <span class="plainlinks">[https://godoc.org/golang.org/x/crypto/nacl/secretbox nacl secretbox]</span> which in term uses XSalsa20 and Poly1305 to encrypt and authenticate your configuration with secret-key cryptography. The password is SHA-256 hashed, which produces the key for secretbox. The hashed password is not stored.<br />
<br />
While this provides very good security, we do not recommend storing your encrypted rclone configuration in public, if it contains sensitive information, maybe except if you use a very strong password.<br />
<br />
If it is safe in your environment, you can set the '''RCLONE_CONFIG_PASS''' environment variable to contain your password, in which case it will be used for decrypting the configuration.<br />
<br />
If you are running rclone inside a script, you might want to disable password prompts. To do that, pass the parameter '''--ask-password=false''' to rclone. This will make rclone fail instead of asking for a password, if '''RCLONE_CONFIG_PASS''' doesn’t contain a valid password.<br />
<br />
==Developer Options==<br />
<br />
These options are useful when developing or debugging rclone. There are also some more remote specific options which aren’t documented here which are used for testing. These start with remote name eg '''--drive-test-option''' - see the docs for the remote in question.<br />
<br />
===–cpuprofile=FILE===<br />
<br />
Write CPU profile to file. This can be analysed with '''go tool pprof'''.<br />
<br />
===–dump-bodies===<br />
<br />
Dump HTTP headers and bodies - may contain sensitive info. Can be very verbose. Useful for debugging only.<br />
<br />
===–dump-filters===<br />
<br />
Dump the filters to the output. Useful to see exactly what include and exclude options are filtering on.<br />
<br />
===–dump-headers===<br />
<br />
Dump HTTP headers - may contain sensitive info. Can be very verbose. Useful for debugging only.<br />
<br />
===–memprofile=FILE===<br />
<br />
Write memory profile to file. This can be analysed with '''go tool pprof'''.<br />
<br />
===–no-check-certificate=true/false===<br />
<br />
'''--no-check-certificate''' controls whether a client verifies the server’s certificate chain and host name. If '''--no-check-certificate''' is true, TLS accepts any certificate presented by the server and any host name in that certificate. In this mode, TLS is susceptible to man-in-the-middle attacks.<br />
<br />
This option defaults to '''false'''.<br />
<br />
'''This should be used only for testing.'''<br />
<br />
==Filtering==<br />
<br />
For the filtering options:<br />
<br />
*'''--delete-excluded'''<br />
*'''--filter'''<br />
*'''--filter-from'''<br />
*'''--exclude'''<br />
*'''--exclude-from'''<br />
*'''--include'''<br />
*'''--include-from'''<br />
*'''--files-from'''<br />
*'''--min-size'''<br />
*'''--max-size'''<br />
*'''--min-age'''<br />
*'''--max-age'''<br />
*'''--dump-filters'''<br />
<br />
See the [[RClone - Filtering|filtering section]].<br />
<br />
==Exit Code==<br />
<br />
If any errors occurred during the command, rclone will set a non zero exit code. This allows scripts to detect when rclone operations have failed.<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Configure&diff=159RClone - Configure2016-04-30T19:37:12Z<p>Adam.birds: </p>
<hr />
<div>=Configure=<br />
<br />
First you’ll need to configure rclone. As the object storage systems have quite complicated authentication these are kept in a config file '''.rclone.conf''' in your home directory by default. (You can use the '''--config''' option to choose a different config file.)<br />
<br />
The easiest way to make the config is to run rclone with the config option:<br />
<br />
<pre><br />
rclone config<br />
</pre><br />
<br />
See the following for detailed instructions for:<br />
<br />
* [[RClone - Google Drive|Google Drive]]<br />
* [[RClone - Amazon S3|Amazon S3]]<br />
* [[RClone - Swift|Swift / Rackspace Cloudfiles / Memset Memstore]]<br />
* [[RClone - Dropbox|Dropbox]]<br />
* [[RClone - Google Cloud Storage|Google Cloud Storage]]<br />
* [[RClone - Local FileSystem|Local FileSystem]]<br />
* [[RClone - Amazon Cloud Drive|Amazon Cloud Drive]]<br />
* [[RClone - Backblaze B2|Backblaze B2]]<br />
* [[RClone - Hubic|Hubic]]<br />
* [[RClone - Microsoft One Drive|Microsoft One Drive]]<br />
* [[RClone - Yandex Disk|Yandex Disk]]<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Configure&diff=158RClone - Configure2016-04-30T19:35:58Z<p>Adam.birds: Created page with "=Configure= First you’ll need to configure rclone. As the object storage systems have quite complicated authentication these are kept in a config file '''.rclone.conf''' in..."</p>
<hr />
<div>=Configure=<br />
<br />
First you’ll need to configure rclone. As the object storage systems have quite complicated authentication these are kept in a config file '''.rclone.conf''' in your home directory by default. (You can use the '''--config''' option to choose a different config file.)<br />
<br />
The easiest way to make the config is to run rclone with the config option:<br />
<br />
<pre><br />
rclone config<br />
</pre><br />
<br />
See the following for detailed instructions for:<br />
<br />
* [[RClone - Google Drive|Google Drive]]<br />
* [[RClone - Amazon S3|Amazon S3]]<br />
* [[RClone - Swift|Swift / Rackspace Cloudfiles / Memset Memstore]]<br />
* [[RClone - Dropbox|Dropbox]]<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Backblaze_B2&diff=156RClone - Backblaze B22016-04-30T19:33:34Z<p>Adam.birds: Adam.birds moved page RClone - BackBlaze B2 to RClone - Backblaze B2</p>
<hr />
<div>=Backblaze B2=<br />
<br />
B2 is <span class="plainlinks">[https://www.backblaze.com/b2/ Backblaze’s cloud storage system]</span>.<br />
<br />
Paths are specified as '''remote:bucket''' (or '''remote:''' for the '''lsd''' command.) You may put subdirectories in too, eg '''remote:bucket/path/to/dir'''.<br />
<br />
==Setup==<br />
<br />
Here is an example of making a b2 configuration. First run:<br />
<br />
<pre><br />
rclone config<br />
</pre><br />
<br />
This will guide you through an interactive setup process. You will need your account number (a short hex number) and key (a long hex number) which you can get from the b2 control panel.<br />
<br />
<pre><br />
No remotes found - make a new one<br />
n) New remote<br />
q) Quit config<br />
n/q> n<br />
name> remote<br />
Type of storage to configure.<br />
Choose a number from below, or type in your own value<br />
1 / Amazon Cloud Drive<br />
\ "amazon cloud drive"<br />
2 / Amazon S3 (also Dreamhost, Ceph)<br />
\ "s3"<br />
3 / Backblaze B2<br />
\ "b2"<br />
4 / Dropbox<br />
\ "dropbox"<br />
5 / Google Cloud Storage (this is not Google Drive)<br />
\ "google cloud storage"<br />
6 / Google Drive<br />
\ "drive"<br />
7 / Hubic<br />
\ "hubic"<br />
8 / Local Disk<br />
\ "local"<br />
9 / Microsoft OneDrive<br />
\ "onedrive"<br />
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)<br />
\ "swift"<br />
11 / Yandex Disk<br />
\ "yandex"<br />
Storage> 3<br />
Account ID<br />
account> 123456789abc<br />
Application Key<br />
key> 0123456789abcdef0123456789abcdef0123456789<br />
Endpoint for the service - leave blank normally.<br />
endpoint> <br />
Remote config<br />
--------------------<br />
[remote]<br />
account = 123456789abc<br />
key = 0123456789abcdef0123456789abcdef0123456789<br />
endpoint = <br />
--------------------<br />
y) Yes this is OK<br />
e) Edit this remote<br />
d) Delete this remote<br />
y/e/d> y<br />
</pre><br />
<br />
This remote is called '''remote''' and can now be used like this:<br />
<br />
See all buckets:<br />
<br />
<pre><br />
rclone lsd remote:<br />
</pre><br />
<br />
Make a new bucket<br />
<br />
<pre><br />
Make a new bucket<br />
</pre><br />
<br />
<pre><br />
rclone mkdir remote:bucket<br />
</pre><br />
<br />
List the contents of a bucket:<br />
<br />
<pre><br />
rclone ls remote:bucket<br />
</pre><br />
<br />
Sync '''/home/local/directory''' to the remote bucket, deleting any excess files in the bucket:<br />
<br />
<pre><br />
rclone sync /home/local/directory remote:bucket<br />
</pre><br />
<br />
==Modified Time==<br />
<br />
The modified time is stored as metadata on the object as '''X-Bz-Info-src_last_modified_millis''' as milliseconds since 1970-01-01 in the Backblaze standard. Other tools should be able to use this as a modified time.<br />
<br />
Modified times are used in syncing and are fully supported except in the case of updating a modification time on an existing object. In this case the object will be uploaded again as B2 doesn’t have an API method to set the modification time independent of doing an upload.<br />
<br />
==SHA1 checksums==<br />
<br />
The SHA1 checksums of the files are checked on upload and download and will be used in the syncing process. You can use the '''--checksum''' flag.<br />
<br />
==Versions==<br />
<br />
When rclone uploads a new version of a file it creates a <span class="plainlinks">[https://www.backblaze.com/b2/docs/file_versions.html new version of it]</span>. Likewise when you delete a file, the old version will still be available.<br />
<br />
The old versions of files are visible in the B2 web interface, but not via rclone yet.<br />
<br />
Rclone doesn’t provide any way of managing old versions (downloading them or deleting them) at the moment. When you '''purge''' a bucket, all the old versions will be deleted.<br />
<br />
==Transfers==<br />
<br />
Backblaze recommends that you do lots of transfers simultaneously for maximum speed. In tests from my SSD equiped laptop the optimum setting is about '''--transfers 32''' though higher numbers may be used for a slight speed improvement. The optimum number for you may vary depending on your hardware, how big the files are, how much you want to load your computer, etc. The default of '''--transfers 4''' is definitely too low for Backblaze B2 though.<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Backblaze_B2&diff=155RClone - Backblaze B22016-04-30T19:32:57Z<p>Adam.birds: Adam.birds moved page RClone - BlackBlaze B2 to RClone - BackBlaze B2 without leaving a redirect</p>
<hr />
<div>=Backblaze B2=<br />
<br />
B2 is <span class="plainlinks">[https://www.backblaze.com/b2/ Backblaze’s cloud storage system]</span>.<br />
<br />
Paths are specified as '''remote:bucket''' (or '''remote:''' for the '''lsd''' command.) You may put subdirectories in too, eg '''remote:bucket/path/to/dir'''.<br />
<br />
==Setup==<br />
<br />
Here is an example of making a b2 configuration. First run:<br />
<br />
<pre><br />
rclone config<br />
</pre><br />
<br />
This will guide you through an interactive setup process. You will need your account number (a short hex number) and key (a long hex number) which you can get from the b2 control panel.<br />
<br />
<pre><br />
No remotes found - make a new one<br />
n) New remote<br />
q) Quit config<br />
n/q> n<br />
name> remote<br />
Type of storage to configure.<br />
Choose a number from below, or type in your own value<br />
1 / Amazon Cloud Drive<br />
\ "amazon cloud drive"<br />
2 / Amazon S3 (also Dreamhost, Ceph)<br />
\ "s3"<br />
3 / Backblaze B2<br />
\ "b2"<br />
4 / Dropbox<br />
\ "dropbox"<br />
5 / Google Cloud Storage (this is not Google Drive)<br />
\ "google cloud storage"<br />
6 / Google Drive<br />
\ "drive"<br />
7 / Hubic<br />
\ "hubic"<br />
8 / Local Disk<br />
\ "local"<br />
9 / Microsoft OneDrive<br />
\ "onedrive"<br />
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)<br />
\ "swift"<br />
11 / Yandex Disk<br />
\ "yandex"<br />
Storage> 3<br />
Account ID<br />
account> 123456789abc<br />
Application Key<br />
key> 0123456789abcdef0123456789abcdef0123456789<br />
Endpoint for the service - leave blank normally.<br />
endpoint> <br />
Remote config<br />
--------------------<br />
[remote]<br />
account = 123456789abc<br />
key = 0123456789abcdef0123456789abcdef0123456789<br />
endpoint = <br />
--------------------<br />
y) Yes this is OK<br />
e) Edit this remote<br />
d) Delete this remote<br />
y/e/d> y<br />
</pre><br />
<br />
This remote is called '''remote''' and can now be used like this:<br />
<br />
See all buckets:<br />
<br />
<pre><br />
rclone lsd remote:<br />
</pre><br />
<br />
Make a new bucket<br />
<br />
<pre><br />
Make a new bucket<br />
</pre><br />
<br />
<pre><br />
rclone mkdir remote:bucket<br />
</pre><br />
<br />
List the contents of a bucket:<br />
<br />
<pre><br />
rclone ls remote:bucket<br />
</pre><br />
<br />
Sync '''/home/local/directory''' to the remote bucket, deleting any excess files in the bucket:<br />
<br />
<pre><br />
rclone sync /home/local/directory remote:bucket<br />
</pre><br />
<br />
==Modified Time==<br />
<br />
The modified time is stored as metadata on the object as '''X-Bz-Info-src_last_modified_millis''' as milliseconds since 1970-01-01 in the Backblaze standard. Other tools should be able to use this as a modified time.<br />
<br />
Modified times are used in syncing and are fully supported except in the case of updating a modification time on an existing object. In this case the object will be uploaded again as B2 doesn’t have an API method to set the modification time independent of doing an upload.<br />
<br />
==SHA1 checksums==<br />
<br />
The SHA1 checksums of the files are checked on upload and download and will be used in the syncing process. You can use the '''--checksum''' flag.<br />
<br />
==Versions==<br />
<br />
When rclone uploads a new version of a file it creates a <span class="plainlinks">[https://www.backblaze.com/b2/docs/file_versions.html new version of it]</span>. Likewise when you delete a file, the old version will still be available.<br />
<br />
The old versions of files are visible in the B2 web interface, but not via rclone yet.<br />
<br />
Rclone doesn’t provide any way of managing old versions (downloading them or deleting them) at the moment. When you '''purge''' a bucket, all the old versions will be deleted.<br />
<br />
==Transfers==<br />
<br />
Backblaze recommends that you do lots of transfers simultaneously for maximum speed. In tests from my SSD equiped laptop the optimum setting is about '''--transfers 32''' though higher numbers may be used for a slight speed improvement. The optimum number for you may vary depending on your hardware, how big the files are, how much you want to load your computer, etc. The default of '''--transfers 4''' is definitely too low for Backblaze B2 though.<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Overview&diff=154RClone - Overview2016-04-30T19:23:07Z<p>Adam.birds: </p>
<hr />
<div>=Overview of Cloud Storage Systems=<br />
<br />
Each cloud storage system is slighly different. Rclone attempts to provide a unified interface to them, but some underlying differences show through.<br />
<br />
==Features==<br />
<br />
Here is an overview of the major features of each cloud storage system:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name<br />
! Hash<br />
! ModTime<br />
! Case Insensitive<br />
! Duplicate Files<br />
|-<br />
| Google Drive<br />
| MD5<br />
| Yes<br />
| No<br />
| Yes<br />
|-<br />
| Amazon S3<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Openstack Swift<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Dropbox<br />
| -<br />
| No<br />
| Yes<br />
| No<br />
|-<br />
| Google Cloud Storage<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Amazon Cloud Drive<br />
| MD5<br />
| No<br />
| Yes<br />
| No<br />
|-<br />
| Microsoft One Drive<br />
| SHA1<br />
| Yes<br />
| Yes<br />
| No<br />
|-<br />
| Hubic<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Backblaze B2<br />
| SHA1<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Yandex Disk<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| The Local Filesystem<br />
| All<br />
| Yes<br />
| Depends<br />
| No<br />
|}<br />
<br />
<br />
==Hash==<br />
<br />
The cloud storage system supports various hash types of the objects.<br />
The hashes are used when transferring data as an integrity check and can be specifically used with the '''--checksum''' flag in syncs and in the '''check''' command.<br />
<br />
To use the checksum checks between filesystems they must support a common hash type.<br />
<br />
==ModTime==<br />
<br />
The cloud storage system supports setting modification times on objects. If it does then this enables a using the modification times as part of the sync. If not then only the size will be checked by default, though the MD5SUM can be checked with the '''--checksum''' flag.<br />
<br />
All cloud storage systems support some kind of date on the object and these will be set when transferring from the cloud storage system.<br />
<br />
==Case Insensitive==<br />
<br />
If a cloud storage systems is case sensitive then it is possible to have two files which differ only in case, eg '''file.txt''' and '''FILE.txt'''. If a cloud storage system is case insensitive then that isn’t possible.<br />
<br />
This can cause problems when syncing between a case insensitive system and a case sensitive system. The symptom of this is that no matter how many times you run the sync it never completes fully.<br />
<br />
The local filesystem may or may not be case sensitive depending on OS.<br />
<br />
Windows - usually case insensitive, though case is preserved<br />
OSX - usually case insensitive, though it is possible to format case sensitive<br />
Linux - usually case sensitive, but there are case insensitive file systems (eg FAT formatted USB keys)<br />
Most of the time this doesn’t cause any problems as people tend to avoid files whose name differs only by case even on case sensitive systems.<br />
<br />
==Duplicate Files==<br />
<br />
If a cloud storage system allows duplicate files then it can have two objects with the same name.<br />
<br />
This confuses rclone greatly when syncing - use the '''rclone dedupe''' command to rename or remove duplicates.<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=RClone_-_Overview&diff=153RClone - Overview2016-04-30T19:22:14Z<p>Adam.birds: Created page with "=Overview of Cloud Storage Systems= Each cloud storage system is slighly different. Rclone attempts to provide a unified interface to them, but some underlying differences sh..."</p>
<hr />
<div>=Overview of Cloud Storage Systems=<br />
<br />
Each cloud storage system is slighly different. Rclone attempts to provide a unified interface to them, but some underlying differences show through.<br />
<br />
=Features=<br />
<br />
Here is an overview of the major features of each cloud storage system:<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name<br />
! Hash<br />
! ModTime<br />
! Case Insensitive<br />
! Duplicate Files<br />
|-<br />
| Google Drive<br />
| MD5<br />
| Yes<br />
| No<br />
| Yes<br />
|-<br />
| Amazon S3<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Openstack Swift<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Dropbox<br />
| -<br />
| No<br />
| Yes<br />
| No<br />
|-<br />
| Google Cloud Storage<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Amazon Cloud Drive<br />
| MD5<br />
| No<br />
| Yes<br />
| No<br />
|-<br />
| Microsoft One Drive<br />
| SHA1<br />
| Yes<br />
| Yes<br />
| No<br />
|-<br />
| Hubic<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Backblaze B2<br />
| SHA1<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| Yandex Disk<br />
| MD5<br />
| Yes<br />
| No<br />
| No<br />
|-<br />
| The Local Filesystem<br />
| All<br />
| Yes<br />
| Depends<br />
| No<br />
|}<br />
<br />
<br />
=Hash=<br />
<br />
The cloud storage system supports various hash types of the objects.<br />
The hashes are used when transferring data as an integrity check and can be specifically used with the '''--checksum''' flag in syncs and in the '''check''' command.<br />
<br />
To use the checksum checks between filesystems they must support a common hash type.<br />
<br />
=ModTime=<br />
<br />
The cloud storage system supports setting modification times on objects. If it does then this enables a using the modification times as part of the sync. If not then only the size will be checked by default, though the MD5SUM can be checked with the '''--checksum''' flag.<br />
<br />
All cloud storage systems support some kind of date on the object and these will be set when transferring from the cloud storage system.<br />
<br />
==Case Insensitive==<br />
<br />
If a cloud storage systems is case sensitive then it is possible to have two files which differ only in case, eg '''file.txt''' and '''FILE.txt'''. If a cloud storage system is case insensitive then that isn’t possible.<br />
<br />
This can cause problems when syncing between a case insensitive system and a case sensitive system. The symptom of this is that no matter how many times you run the sync it never completes fully.<br />
<br />
The local filesystem may or may not be case sensitive depending on OS.<br />
<br />
Windows - usually case insensitive, though case is preserved<br />
OSX - usually case insensitive, though it is possible to format case sensitive<br />
Linux - usually case sensitive, but there are case insensitive file systems (eg FAT formatted USB keys)<br />
Most of the time this doesn’t cause any problems as people tend to avoid files whose name differs only by case even on case sensitive systems.<br />
<br />
==Duplicate Files=<br />
<br />
If a cloud storage system allows duplicate files then it can have two objects with the same name.<br />
<br />
This confuses rclone greatly when syncing - use the '''rclone dedupe''' command to rename or remove duplicates.<br />
<br />
[[Category:RClone]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Prestashop&diff=152Category:Prestashop2016-04-30T17:29:29Z<p>Adam.birds: Created page with "Category:Contents"</p>
<hr />
<div>[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Prestashop_Nginx_Config&diff=151Prestashop Nginx Config2016-04-30T17:29:13Z<p>Adam.birds: </p>
<hr />
<div>'''Below is the Nginx config for Prestashop:'''<br />
<br />
<pre><br />
upstream examplecombackend {<br />
server unix:/var/run/php-fcgi-examplecom.sock;<br />
}<br />
<br />
server {<br />
<br />
listen 80;<br />
server_name example.com;<br />
root /var/www/vhosts/example.com/htdocs;<br />
index index.php index.html;<br />
<br />
access_log /var/www/vhosts/example.com/access.log;<br />
error_log /var/www/vhosts/example.com/error.log warn;<br />
<br />
location / {<br />
try_files $uri $uri/ @handler;<br />
index index.php index.html;<br />
}<br />
<br />
# PrestaShop rewrite rules<br />
rewrite ^/([a-z0-9]+)-([a-z0-9]+)(-[_a-zA-Z0-9-]*)/([_a-zA-Z0-9-]*).jpg$ /img/p/$1-$2$3.jpg last;<br />
rewrite ^/([0-9]+)-([0-9]+)/([_a-zA-Z0-9-]*).jpg$ /img/p/$1-$2.jpg last;<br />
rewrite ^/([0-9]+)(-[_a-zA-Z0-9-]*)/([_a-zA-Z0-9-]*).jpg$ /img/c/$1$2.jpg last;<br />
rewrite "^/lang-([a-z]{2})/([a-zA-Z0-9-]*)/([0-9]+)-([a-zA-Z0-9-]*).html(.*)$" /product.php?id_product=$3&isolang=$1$5 last;<br />
rewrite "^/lang-([a-z]{2})/([0-9]+)-([a-zA-Z0-9-]*).html(.*)$" /product.php?id_product=$2&isolang=$1$4 last;<br />
rewrite "^/lang-([a-z]{2})/([0-9]+)-([a-zA-Z0-9-]*)(.*)$" /category.php?id_category=$2&isolang=$1 last;<br />
rewrite ^/([a-zA-Z0-9-]*)/([0-9]+)-([a-zA-Z0-9-]*).html(.*)$ /product.php?id_product=$2$4 last;<br />
rewrite ^/([0-9]+)-([a-zA-Z0-9-]*).html(.*)$ /product.php?id_product=$1$3 last;<br />
rewrite ^/([0-9]+)-([a-zA-Z0-9-]*)(.*)$ /category.php?id_category=$1 last;<br />
rewrite ^/content/([0-9]+)-([a-zA-Z0-9-]*)(.*)$ /cms.php?id_cms=$1 last;<br />
rewrite ^/([0-9]+)__([a-zA-Z0-9-]*)(.*)$ /supplier.php?id_supplier=$1$3 last;<br />
rewrite ^/([0-9]+)_([a-zA-Z0-9-]*)(.*)$ /manufacturer.php?id_manufacturer=$1$3 last;<br />
rewrite "^/lang-([a-z]{2})/(.*)$" /$2?isolang=$1 last;<br />
<br />
# static file cache configuration<br />
location ~* .(gif)$ {<br />
expires 2592000s;<br />
}<br />
location ~* .(jpeg|jpg)$ {<br />
expires 2592000s;<br />
}<br />
location ~* .(png)$ {<br />
expires 2592000s;<br />
}<br />
location ~* .(css)$ {<br />
expires 604800s;<br />
}<br />
location ~* .(js)$ {<br />
expires 604800s;<br />
}<br />
location ~* .(js)$ {<br />
expires 604800s;<br />
}<br />
location ~* .(ico)$ {<br />
expires 31536000s;<br />
}<br />
<br />
# php-fpm configuration<br />
location ~ .php$ {<br />
root /var/www/vhosts/example.com/htdocs;<br />
try_files $uri =404;<br />
fastcgi_pass examplecombackend;<br />
fastcgi_index index.php;<br />
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br />
include fastcgi_params;<br />
fastcgi_buffer_size 128k;<br />
fastcgi_buffers 256 4k;<br />
fastcgi_busy_buffers_size 256k;<br />
fastcgi_temp_file_write_size 256k;<br />
}<br />
}<br />
</pre><br />
<br />
[[Category:Prestashop]]<br />
[[Category:Nginx]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Moodle&diff=150Category:Moodle2016-04-30T17:28:33Z<p>Adam.birds: Created page with "Category:Moodle"</p>
<hr />
<div>[[Category:Moodle]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Moodle_Nginx_Config&diff=149Moodle Nginx Config2016-04-30T17:28:17Z<p>Adam.birds: </p>
<hr />
<div>'''Here is the Nginx Config for Moodle:'''<br />
<br />
<pre><br />
upstream examplecombackend {<br />
server unix:/var/run/php-fcgi-examplecom.sock;<br />
}<br />
<br />
server {<br />
listen 80;<br />
listen 443 ssl;<br />
server_name example.com;<br />
root /var/www/vhosts/example.com/htdocs;<br />
<br />
ssl_certificate /etc/nginx/ssl/example.com.crt;<br />
ssl_certificate_key /etc/nginx/ssl/example.com.key;<br />
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;<br />
ssl_ciphers RC4-SHA:!HIGH:!ADH;<br />
<br />
location / {<br />
index index.html index.php;<br />
try_files $uri $uri/ @handler;<br />
expires 30d;<br />
}<br />
<br />
location /app/ { deny all; }<br />
location /includes/ { deny all; }<br />
location /media/downloadable/ { deny all; }<br />
location /pkginfo/ { deny all; }<br />
location /report/config.xml { deny all; }<br />
location /var/ { deny all; }<br />
<br />
location /. {<br />
return 404;<br />
}<br />
<br />
location @handler {<br />
rewrite / /index.php;<br />
}<br />
<br />
location /dataroot/ {<br />
internal;<br />
alias /var/www/vhosts/example.com/moodledata/;<br />
}<br />
<br />
include "ssl_offloading.inc";<br />
<br />
location ~ [^/]\.php(/|$) {<br />
fastcgi_split_path_info ^(.+\.php)(/.+)$;<br />
fastcgi_index index.php;<br />
fastcgi_pass examplecombackend;<br />
include fastcgi_params;<br />
fastcgi_param PATH_INFO $fastcgi_path_info;<br />
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br />
}<br />
<br />
}<br />
</pre><br />
<br />
[[Category:Moodle]]<br />
[[Category:Nginx]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Moodle_Initial_Script&diff=148Moodle Initial Script2016-04-30T17:27:52Z<p>Adam.birds: </p>
<hr />
<div>'''Here is the Initial Script for the Moodle Stack:'''<br />
<br />
<pre><br />
#! /bin/bash<br />
####<br />
## INITIAL.SH<br />
####<br />
# ToDo<br />
# tcp window size<br />
# cherokee and stuff<br />
# at job mail support for tuning checkup<br />
# this stuff:<br />
# echo deadline > /sys/block/sda/queue/scheduler<br />
# echo noop > /sys/block/sda/queue/scheduler<br />
# cat /proc/sys/vm/swappiness<br />
# echo 0 > cat /proc/sys/vm/swappiness<br />
# sysctl -p<br />
# ktune<br />
# compcache<br />
# Dane tmpfs init<br />
# chef?!?<br />
# disable repos<br />
# gzip_vary on; https://developers.google.com/speed/docs/best-practices/caching?hl=sv#LeverageProxyCaching<br />
# sed '-' out of dbname<br />
###<br />
<br />
#### ####<br />
## Edit stuff below this point if you don't want defaults ##<br />
#### ####<br />
<br />
#Write your own domain in, just leave out the www<br />
#Can be left blank to use example.com as a standin<br />
DOMAIN=""<br />
#IP can be ext, int or write it in yourself<br />
IP="ext"<br />
#Change SSL to "yes" if you want the nginx SSL section including<br />
SSL="yes"<br />
<br />
#### ####<br />
## Don't edit stuff below here if you want the script to work ##<br />
#### ####<br />
<br />
unset HISTFILE<br />
<br />
###<br />
# Quick bc install for some of the vars<br />
###<br />
<br />
yum install bc at -y<br />
service atd start<br />
chkconfig atd on<br />
<br />
# Other utils that don't come as default<br />
yum install tcpdump lsof wget telnet -y<br />
<br />
####<br />
# Also pwgen<br />
####<br />
yum install epel-release -y<br />
yum install pwgen --enablerepo=epel -y<br />
<br />
###<br />
# Variables 'n' shit<br />
###<br />
<br />
if [ -z ${DOMAIN} ]<br />
then<br />
DOMAIN="example.com"<br />
fi<br />
<br />
if [ "$IP" = "int" ]<br />
then<br />
# '-m1' to return first match only<br />
IP=`ip a|egrep -m1 '(172\.|192.168\.|10\.0\.)'|awk '{print $2}'|sed -e 's_/.*__'`<br />
elif [ "$IP" = "ext" ]<br />
then<br />
IP=`curl -s icanhazip.com`<br />
fi<br />
<br />
ARCH=`arch`<br />
THREADCON=$(echo 2*$(grep -c "processor" /proc/cpuinfo)+2|bc)<br />
NGINXWORKER=$(grep -c "processor" /proc/cpuinfo)<br />
POOLSIZE=$(echo $(echo $(grep MemTot /proc/meminfo |awk '{print $2}')/2|bc)k)<br />
DOMNODOT=`echo $DOMAIN| sed 's/\.//g'`<br />
BACKEND=$(echo $(echo $DOMNODOT)backend)<br />
USERPASS=`pwgen -cn1`<br />
MOODLEUSER="admin-$(cat /dev/urandom | tr -dc '0-9a-zA-Z' | head -c 2)"<br />
MOODLEPASS=`pwgen -cn1`<br />
MYSQL=`echo $DOMNODOT|sed -n 's/^\(......\).*/\1/p'`<br />
MYSQL="${MYSQL}$(cat /dev/urandom | tr -dc '0-9a-zA-Z' | head -c 4)"<br />
MYSQLPASS=`pwgen -cn1`<br />
MYSQLROOTPASS=`pwgen -cn1`<br />
USERNAME=$(echo $DOMNODOT|cut -c 1-15)<br />
USERNAME="${USERNAME}$(cat /dev/urandom|tr -dc '0-9a-z'| head -c 6)"<br />
DBNAME=`echo $DOMNODOT|sed 's/-//g'`<br />
<br />
#install atomic<br />
wget -q -O - http://www.atomicorp.com/installers/atomic | sed -e '/check_input "Enable repo/d' -e 's/query=$INPUTTEXT/query="yes"/g'|NON_INT=1 sh<br />
sed -i 's/gpgcheck = 1/gpgcheck = 0/g' /etc/yum.repos.d/atomic.repo<br />
<br />
if [[ "`grep "release 7" /etc/redhat-release`" =~ "release 7" ]]; then<br />
yum -y install https://dl.iuscommunity.org/pub/ius/stable/CentOS/7/x86_64/ius-release-1.0-14.ius.centos7.noarch.rpm<br />
yum -y install php56u-gd php56u-cli php56u-devel php56u-fpm php56u-process php56u-soap php56u-pecl-jsonc php56u-pdo php56u php56u-mysqlnd php56u-gmp php56u-opcache php56u-pear php56u-common php56u-xmlrpc php56u-odbc php56u-mbstring php56u-ldap php56u-pecl-jsonc-devel php56u-xml php56u-pgsql php56u-intl php56u-mcrypt openldap-devel openldap-clients openldap-servers openldap openldap-servers-sql<br />
else<br />
yum -y install https://dl.iuscommunity.org/pub/ius/stable/CentOS/6/x86_64/ius-release-1.0-14.ius.centos6.noarch.rpm<br />
yum -y install php56u-gd php56u-cli php56u-devel php56u-fpm php56u-process php56u-soap php56u-pecl-jsonc php56u-pdo php56u php56u-mysqlnd php56u-gmp php56u-opcache php56u-pear php56u-common php56u-xmlrpc php56u-odbc php56u-mbstring php56u-ldap php56u-pecl-jsonc-devel php56u-xml php56u-pgsql php56u-intl php56u-mcrypt openldap-devel openldap-clients openldap-servers openldap openldap-servers-sql<br />
fi<br />
<br />
<br />
# todo install epel nicer? from their epel-release rpm?<br />
sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/epel.repo<br />
<br />
#yum -y install php-mcrypt-5.3* --enablerepo=epel<br />
<br />
<br />
###<br />
# Check httpd isn't going to mess stuff up<br />
###<br />
<br />
service httpd stop<br />
chkconfig httpd off<br />
<br />
###<br />
# Same with cherokee<br />
###<br />
<br />
rpm -e --nodeps cherokee<br />
<br />
###<br />
# misc atop vim screen install<br />
###<br />
<br />
yum install atop vim-enhanced screen -y<br />
chkconfig atop on<br />
service atop start<br />
<br />
###<br />
# Time to get memcached on<br />
###<br />
yum install memcached -y<br />
service memcached start<br />
chkconfig memcached on<br />
<br />
###<br />
# And then redis...<br />
###<br />
yum -y install redis<br />
service redis start<br />
chkconfig redis on<br />
<br />
###<br />
# Now for nginx<br />
###<br />
<br />
echo "<br />
[nginx]<br />
name=nginx repo<br />
baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/<br />
gpgcheck=0<br />
enabled=1<br />
" >> /etc/yum.repos.d/nginx.repo<br />
yum install nginx -y<br />
chkconfig nginx on<br />
<br />
###<br />
# Install apc/memcache<br />
# Also zlib-devel for memcache<br />
###<br />
<br />
yum install gcc make zlib-devel pcre-devel -y<br />
# need to install memcache-beta to make sure its v3.0.6 for memcache<br />
printf "\n" |pecl install memcache-beta<br />
# redis module, just in case<br />
printf "\n" |pecl install redis<br />
<br />
echo "<br />
extension=memcache.so<br />
" > /etc/php.d/memcache.ini<br />
<br />
echo "<br />
extension=redis.so<br />
" >> /etc/php.d/redis.ini<br />
<br />
<br />
###<br />
# PHP tweaks<br />
###<br />
<br />
sed -e "s_;date.timezone =_date.timezone = \"Europe/London\"_g" \<br />
-e "s/memory_limit = 128M/memory_limit = 512M/" \<br />
-e "s/max_execution_time = 30/max_execution_time = 600/" \<br />
-e "s/session.save_handler = files/session.save_handler = memcache/" \<br />
-e '/session.save_path = "\//c\session.save_path = "tcp:\/\/127.0.0.1:11211"' \<br />
-e "s/short_open_tag = Off/short_open_tag = On/"\<br />
/etc/php.ini > /etc/php.ini.new && mv -f /etc/php.ini.new /etc/php.ini<br />
<br />
#fixpath for moodle<br />
sed -i 's/;cgi.fix_pathinfo=1/cgi.fix_pathinfo = 0/g' /etc/php.ini<br />
<br />
<br />
###<br />
# For testing purposes, installing mysql first<br />
###<br />
<br />
#yum install mysql mysql-server -y<br />
#service mysqld start<br />
<br />
###<br />
# Percona Swap-out<br />
###<br />
<br />
yum -y install http://www.percona.com/downloads/percona-release/redhat/0.1-3/percona-release-0.1-3.noarch.rpm<br />
service mysqld stop<br />
rpm -e --nodeps mysql mysql-server<br />
cp -rp /var/lib/mysql /var/lib/mysql.copy<br />
rm -rf /var/lib/mysql<br />
yum install Percona-Server-client-56 Percona-Server-server-56 Percona-Server-shared-compat -y<br />
service mysql start<br />
chkconfig mysql on<br />
mysql_upgrade<br />
service mysql stop<br />
<br />
###<br />
# MySQL optimisations<br />
###<br />
<br />
cat /dev/null > /etc/my.cnf<br />
<br />
echo "<br />
[mysqld]<br />
datadir=/var/lib/mysql<br />
socket=/var/lib/mysql/mysql.sock<br />
user=mysql<br />
<br />
# Start of Base Moodle Optimisation<br />
innodb_flush_log_at_trx_commit = 2<br />
innodb_flush_method = O_DIRECT<br />
innodb_file_per_table<br />
innodb_thread_concurrency = $THREADCON<br />
query_cache_size = 128M<br />
query_cache_limit = 8M<br />
thread_cache_size = 64<br />
key_buffer_size = 512M<br />
max_allowed_packet = 64M<br />
table_open_cache = 512<br />
sort_buffer_size = 4M<br />
read_buffer_size = 4M<br />
read_rnd_buffer_size = 2M<br />
myisam_sort_buffer_size = 64M<br />
max_heap_table_size = 128M<br />
tmp_table_size = 128M<br />
query_cache_type = 1<br />
wait_timeout = 300<br />
max_connections = 200<br />
innodb_buffer_pool_size = $POOLSIZE<br />
# End of Base Moodle Optimisation<br />
<br />
[mysqld_safe]<br />
log-error=/var/log/mysqld.log<br />
pid-file=/var/run/mysqld/mysqld.pid<br />
<br />
[client]<br />
socket=/var/lib/mysql/mysql.sock<br />
" >> /etc/my.cnf<br />
<br />
service mysql start<br />
<br />
###<br />
# nginx shit<br />
###<br />
<br />
cat /dev/null > /etc/nginx/nginx.conf<br />
<br />
echo "<br />
user nginx;<br />
worker_processes $NGINXWORKER;<br />
error_log /var/log/nginx/error.log;<br />
pid /var/run/nginx.pid;<br />
<br />
events {<br />
worker_connections 1024;<br />
multi_accept on;<br />
use epoll;<br />
}<br />
<br />
http {<br />
include /etc/nginx/mime.types;<br />
default_type application/octet-stream;<br />
server_tokens on;<br />
<br />
log_format main '\$remote_addr - \$remote_user [\$time_local] \"\$request \"'<br />
'\$status \$body_bytes_sent \"\$http_referer\" '<br />
'\"\$http_user_agent\" \"\$http_x_forwarded_for\"'<br />
'[RT:\$request_time] [C:\$connection]';<br />
access_log /var/log/nginx/access.log main buffer=32k flush=300;<br />
<br />
sendfile on;<br />
tcp_nopush on;<br />
tcp_nodelay on;<br />
autoindex off;<br />
map \$scheme \$fastcgi_https { ## Detect when HTTPS is used<br />
default off;<br />
https on;<br />
}<br />
<br />
# Dirty fix as some Magento scripts (admin) take up-to 10 minutes!<br />
fastcgi_read_timeout 600;<br />
fastcgi_send_timeout 600;<br />
<br />
#big header fixes<br />
proxy_buffers 8 16k;<br />
proxy_buffer_size 32k;<br />
fastcgi_buffers 16 16k;<br />
fastcgi_buffer_size 32k;<br />
<br />
client_body_buffer_size 10K;<br />
client_header_buffer_size 1k;<br />
client_max_body_size 15m;<br />
large_client_header_buffers 2 1k;<br />
<br />
keepalive_timeout 60;<br />
<br />
gzip on;<br />
gzip_comp_level 9;<br />
gzip_proxied any;<br />
gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;<br />
gzip_min_length 20;<br />
<br />
open_file_cache max=10000 inactive=5m;<br />
open_file_cache_valid 2m;<br />
open_file_cache_min_uses 1;<br />
open_file_cache_errors on;<br />
<br />
# Load config files from the /etc/nginx/conf.d directory<br />
include /etc/nginx/conf.d/*.conf;<br />
<br />
}<br />
" >> /etc/nginx/nginx.conf<br />
<br />
###<br />
# SSL Offloading fix<br />
###<br />
<br />
echo "<br />
if (\$http_x_forwarded_proto = \"https\") { ## Safety net to catch SSL-Offloading<br />
set \$fastcgi_https \"on\";<br />
}<br />
" >> /etc/nginx/ssl_offloading.inc<br />
<br />
<br />
###<br />
# Adding first domain<br />
###<br />
if [ "$SSL" = "no" ]; then<br />
echo "<br />
# Uncomment the server definition below should you wish to<br />
# redirect from $DOMAIN to www.$DOMAIN<br />
#server {<br />
# listen $IP;<br />
# server_name $DOMAIN;<br />
# #rewrite / \$scheme://www.\$host\$request_uri permanent;<br />
#}<br />
<br />
#<br />
# Change this backend name (and the socket pointer)<br />
# as additional virtual hosts are added. This should<br />
# point to the spawn-fcgi wrapper running as the<br />
# appropriate user.<br />
#<br />
upstream $BACKEND {<br />
server unix:/var/run/php-fcgi-$DOMNODOT.sock;<br />
}<br />
<br />
server {<br />
listen $IP:80;<br />
server_name $DOMAIN *.$DOMAIN;<br />
root /var/www/vhosts/$DOMAIN/htdocs;<br />
<br />
location / {<br />
index index.html index.php;<br />
try_files \$uri \$uri/ @handler;<br />
expires 30d;<br />
}<br />
<br />
location /app/ { deny all; }<br />
location /includes/ { deny all; }<br />
location /media/downloadable/ { deny all; }<br />
location /pkginfo/ { deny all; }<br />
location /report/config.xml { deny all; }<br />
location /var/ { deny all; }<br />
<br />
location /. {<br />
return 404;<br />
}<br />
<br />
location @handler {<br />
rewrite / /index.php;<br />
}<br />
<br />
location /dataroot/ {<br />
internal;<br />
alias /var/www/vhosts/$DOMAIN/moodledata/;<br />
}<br />
<br />
include \"ssl_offloading.inc\";<br />
<br />
location ~ [^/]\.php(/|$) {<br />
fastcgi_split_path_info ^(.+\.php)(/.+)$;<br />
fastcgi_index index.php;<br />
fastcgi_pass $BACKEND;<br />
include fastcgi_params;<br />
fastcgi_param PATH_INFO \$fastcgi_path_info;<br />
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;<br />
}<br />
<br />
}<br />
" >> /etc/nginx/conf.d/$DOMAIN.conf<br />
else<br />
echo "<br />
# Uncomment the server definition below should you wish to<br />
# redirect from $DOMAIN to www.$DOMAIN<br />
#server {<br />
# listen $IP;<br />
# server_name $DOMAIN;<br />
# #rewrite / \$scheme://www.\$host\$request_uri permanent;<br />
#}<br />
<br />
#<br />
# Change this backend name (and the socket pointer)<br />
# as additional virtual hosts are added. This should<br />
# point to the spawn-fcgi wrapper running as the<br />
# appropriate user.<br />
#<br />
upstream $BACKEND {<br />
server unix:/var/run/php-fcgi-$DOMNODOT.sock;<br />
}<br />
<br />
server {<br />
listen $IP:80;<br />
listen $IP:443 ssl;<br />
server_name $DOMAIN *.$DOMAIN;<br />
root /var/www/vhosts/$DOMAIN/htdocs;<br />
<br />
ssl_certificate /etc/nginx/ssl/$DOMAIN.crt;<br />
ssl_certificate_key /etc/nginx/ssl/$DOMAIN.key;<br />
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;<br />
ssl_ciphers RC4-SHA:!HIGH:!ADH;<br />
<br />
location / {<br />
index index.html index.php;<br />
try_files \$uri \$uri/ @handler;<br />
expires 30d;<br />
}<br />
<br />
location /app/ { deny all; }<br />
location /includes/ { deny all; }<br />
location /media/downloadable/ { deny all; }<br />
location /pkginfo/ { deny all; }<br />
location /report/config.xml { deny all; }<br />
location /var/ { deny all; }<br />
<br />
location /. {<br />
return 404;<br />
}<br />
<br />
location @handler {<br />
rewrite / /index.php;<br />
}<br />
<br />
location /dataroot/ {<br />
internal;<br />
alias /var/www/vhosts/$DOMAIN/moodledata/;<br />
}<br />
<br />
include \"ssl_offloading.inc\";<br />
<br />
location ~ [^/]\.php(/|$) {<br />
fastcgi_split_path_info ^(.+\.php)(/.+)$;<br />
fastcgi_index index.php;<br />
fastcgi_pass $BACKEND;<br />
include fastcgi_params;<br />
fastcgi_param PATH_INFO \$fastcgi_path_info;<br />
fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name;<br />
}<br />
<br />
}<br />
" >> /etc/nginx/conf.d/$DOMAIN.conf<br />
<br />
mkdir -p /etc/nginx/ssl<br />
<br />
#generate self signed for placeholder<br />
openssl genrsa 2048 > /etc/nginx/ssl/$DOMAIN.key<br />
openssl req -new -key /etc/nginx/ssl/$DOMAIN.key -x509 -days 365 -out /etc/nginx/ssl/$DOMAIN.crt -batch<br />
fi<br />
<br />
mkdir -p /var/www/vhosts/$DOMAIN/htdocs<br />
useradd -d /var/www/vhosts/$DOMAIN -s /sbin/nologin $USERNAME<br />
echo $USERPASS|passwd --stdin $USERNAME<br />
touch /var/www/vhosts/$DOMAIN/phpfpm-slow.log<br />
touch /var/www/vhosts/$DOMAIN/phpfpm-error.log<br />
chown -R $USERNAME:$USERNAME /var/www/vhosts/$DOMAIN<br />
<br />
###<br />
# Clean up nginx<br />
###<br />
<br />
rm -f /etc/nginx/conf.d/default.conf<br />
rm -f /etc/nginx/conf.d/example_ssl.conf<br />
<br />
###<br />
# PHP time<br />
###<br />
<br />
rm -f /etc/php-fpm.d/*<br />
chkconfig php-fpm on<br />
<br />
echo "<br />
[$DOMNODOT]<br />
<br />
listen = '/var/run/php-fcgi-$DOMNODOT.sock'<br />
;listen.backlog = -1<br />
listen.allowed_clients = 127.0.0.1<br />
listen.owner = nginx<br />
listen.group = nginx<br />
;listen.mode = 0666<br />
<br />
user = $USERNAME<br />
group = $USERNAME<br />
<br />
pm = dynamic<br />
pm.max_children = 50<br />
pm.start_servers = 5<br />
pm.min_spare_servers = 5<br />
pm.max_spare_servers = 20<br />
pm.max_requests = 2000<br />
;pm.status_path = /status<br />
<br />
;ping.path = /ping<br />
;ping.response = pong<br />
<br />
;request_terminate_timeout = 0<br />
;request_slowlog_timeout = 0<br />
slowlog = /var/www/vhosts/$DOMAIN/phpfpm-slow.log<br />
<br />
;rlimit_files = 1024<br />
;rlimit_core = 0<br />
<br />
;chroot =<br />
<br />
;chdir = /var/www<br />
<br />
;catch_workers_output = yes<br />
<br />
;env[HOSTNAME] = \$HOSTNAME<br />
;env[PATH] = /usr/local/bin:/usr/bin:/bin<br />
;env[TMP] = /tmp<br />
;env[TMPDIR] = /tmp<br />
;env[TEMP] = /tmp<br />
<br />
php_admin_value[error_log] = /var/www/vhosts/$DOMAIN/phpfpm-error.log<br />
php_admin_flag[log_errors] = on<br />
" >> /etc/php-fpm.d/$DOMNODOT.conf<br />
<br />
# Segfault fix<br />
# Pretty sure this is incredibly irrelevant now, commenting out<br />
#sed -i '/RETVAL=0/ i\export ZEND_DONT_UNLOAD_MODULES=1' /etc/init.d/php-fpm<br />
<br />
###<br />
# Create a default DB<br />
###<br />
<br />
echo "CREATE DATABASE $DBNAME;" | mysql<br />
echo "GRANT ALL PRIVILEGES ON $DBNAME.* TO '$MYSQL'@'localhost' IDENTIFIED BY '$MYSQLPASS';" | mysql<br />
echo "FLUSH PRIVILEGES;" | mysql<br />
<br />
###<br />
# Set the root password for the database<br />
###<br />
<br />
echo "SET PASSWORD FOR root@'localhost' = PASSWORD('$MYSQLROOTPASS');" | mysql<br />
echo "FLUSH PRIVILEGES;" | mysql<br />
<br />
#Set .my.cnf for default login<br />
<br />
touch /root/.my.cnf<br />
cat /dev/null > /root/.my.cnf<br />
echo "<br />
[client]<br />
host = localhost<br />
user = root<br />
password = $MYSQLROOTPASS<br />
" > /root/.my.cnf<br />
mysql_upgrade<br />
<br />
###<br />
# FTP stuff<br />
###<br />
<br />
yum install vsftpd -y<br />
chkconfig vsftpd on<br />
sed -i 's/#chroot_local_user=YES/chroot_local_user=YES/' /etc/vsftpd/vsftpd.conf<br />
sed -i 's/anonymous_enable=YES/anonymous_enable=NO/g' /etc/vsftpd/vsftpd.conf<br />
service vsftpd start<br />
<br />
#disable atomic<br />
sed -i 's/enabled = 1/enabled = 0/' /etc/yum.repos.d/atomic.repo<br />
<br />
###<br />
# Install Moodle<br />
###<br />
<br />
wget -O /tmp/moodle.tgz https://download.moodle.org/download.php/direct/stable30/moodle-latest-30.tgz<br />
tar --strip 1 -C /var/www/vhosts/$DOMAIN/htdocs/ -xzf /tmp/moodle.tgz<br />
mkdir /var/www/vhosts/$DOMAIN/moodledata<br />
chmod 777 /var/www/vhosts/$DOMAIN/moodledata #Sorry. Moodle reccomended. will be a way around it, need to look into it.<br />
chown -R $USERNAME:$USERNAME /var/www/vhosts/$DOMAIN<br />
find /var/www/vhosts/$DOMAIN/htdocs/ -type f -exec chmod 0644 {} \;<br />
find /var/www/vhosts/$DOMAIN/htdocs/ -type d -exec chmod 0755 {} \;<br />
sudo -u $USERNAME /usr/bin/php /var/www/vhosts/$DOMAIN/htdocs/admin/cli/install.php --chmod=0777 --lang=en --wwwroot=https://$DOMAIN --dataroot=/var/www/vhosts/$DOMAIN/moodledata --dbname=$DBNAME --dbuser=$MYSQL --dbpass=$MYSQLPASS --fullname=$DOMAIN --shortname=$DOMAIN --summary=default --adminuser=$MOODLEUSER --adminpass=$MOODLEPASS --adminemail=admin@$DOMAIN --non-interactive --agree-license<br />
<br />
crontab -u $USERNAME -l | { cat; echo "* * * * * /usr/bin/php /var/www/vhosts/$DOMAIN/htdocs/admin/cli/cron.php >/dev/null"; } | crontab -u $USERNAME -<br />
<br />
sed -i "/directoryp/a\$CFG->xsendfile = 'X-Accel-Redirect';\n\$CFG->xsendfilealiases = array(\n '/dataroot/' => \$CFG->dataroot\n);" /var/www/vhosts/$DOMAIN/htdocs/config.php<br />
<br />
<br />
###<br />
# Start it all up<br />
###<br />
<br />
service nginx start<br />
service php-fpm start<br />
<br />
echo "<br />
# $DOMAIN now set up<br />
# Their username is $USERNAME<br />
# Their password is $USERPASS<br />
#<br />
# Their database name is $DBNAME<br />
# Their DB username is $MYSQL<br />
# Their DB password is $MYSQLPASS<br />
<br />
# Their Moodle admin user is $MOODLEUSER<br />
# Their Moodle admin password is $MOODLEPASS<br />
#<br />
# This file will self destruct in 24 hours.<br />
#<br />
# Note: MySQL root password set to $MYSQLROOTPASS<br />
" >> /root/finished$DOMAIN<br />
<br />
cat /root/finished$DOMAIN<br />
echo "rm -f /root/finished$DOMAIN" | at now + 24 hours<br />
<br />
if [ "$SSL" = "yes" ]<br />
then<br />
echo "<br />
# Self signed SSL certificates generated in the following location:<br />
# /etc/nginx/ssl/$DOMNODOT.key<br />
# /etc/nginx/ssl/$DOMNODOT.crt<br />
# So don't forget to replace them with legit ones<br />
" >> /root/finished$DOMAIN<br />
fi<br />
<br />
#Delete self<br />
history -c<br />
rm -f $0<br />
</pre><br />
<br />
[[Category:Moodle]]<br />
[[Category:Nginx]]<br />
[[Category:PHP]]<br />
[[Category:Scripts]]<br />
[[Category:Bash Scripts]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Joomla&diff=147Category:Joomla2016-04-30T17:27:20Z<p>Adam.birds: Created page with "Category:Contents"</p>
<hr />
<div>[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Joomla_Nginx_Config&diff=146Joomla Nginx Config2016-04-30T17:26:58Z<p>Adam.birds: </p>
<hr />
<div>'''Here is the Nginx Config for Joomla:'''<br />
<br />
<pre><br />
upstream examplecombackend {<br />
server unix:/var/run/php-fcgi-examplecom.sock;<br />
}<br />
<br />
server {<br />
listen 80;<br />
server_name example.com;<br />
server_name_in_redirect off;<br />
<br />
access_log /var/www/vhosts/example.com/access.log;<br />
error_log /var/www/vhosts/example.com/error.log warn;<br />
<br />
root /var/www/vhosts/example.com/htdocs;<br />
index index.php index.html index.htm default.html default.htm;<br />
<br />
# Support Clean (aka Search Engine Friendly) URLs<br />
location / {<br />
try_files $uri $uri/ /index.php?$args;<br />
}<br />
<br />
# deny running scripts inside writable directories<br />
location ~* /(images|cache|media|logs|tmp)/.*\.(php|pl|py|jsp|asp|sh|cgi)$ {<br />
return 403;<br />
error_page 403 /403_error.html;<br />
}<br />
<br />
location ~ \.php$ {<br />
fastcgi_pass examplecombackend;<br />
fastcgi_index index.php;<br />
include fastcgi_params;<br />
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;<br />
include /etc/nginx/fastcgi.conf;<br />
}<br />
<br />
# caching of files <br />
location ~* \.(ico|pdf|flv)$ {<br />
expires 1y;<br />
}<br />
<br />
location ~* \.(js|css|png|jpg|jpeg|gif|swf|xml|txt)$ {<br />
expires 14d;<br />
}<br />
<br />
}<br />
</pre><br />
<br />
[[Category:Joomla]]<br />
[[Category:Nginx]]<br />
[[Category:Linux]]<br />
[[Category:Contents]]</div>Adam.birdshttp://techwiki.co.uk/index.php?title=Category:Wordpress&diff=145Category:Wordpress2016-04-30T17:25:58Z<p>Adam.birds: Created page with "Category:Contents"</p>
<hr />
<div>[[Category:Contents]]</div>Adam.birds