tag:blogger.com,1999:blog-80279511244772907532024-03-13T05:10:07.775-07:00Techspot - Databases, web, servers - Philip WuTips and experience about developing websites with various technologiesUnknownnoreply@blogger.comBlogger185125tag:blogger.com,1999:blog-8027951124477290753.post-28187853962513483522021-11-02T22:27:00.000-07:002021-11-03T18:00:43.698-07:00Docker + Logs + Fail2Ban: Originating IP addresses<p> <b><span style="color: #7f6000;">Fail2Ban </span></b>is a great tool for securing our servers and relies heavily on the log files of various services. Recently, we've deployed apps using Docker containers which has been an interesting journey, but there was little in the way of documentation on how to get fail2ban working on logs produced from docker containers. So let's start with the problem:</p><h2 style="text-align: left;"><span style="color: #0b5394; font-size: large;">The problem</span></h2><p>I managed to get Docker to dump mongoDB logs to the host file system under /var/log using the driver syslog following this reference: <a href="https://techroads.org/docker-logging-to-the-local-os-that-works-with-compose-and-rsyslog/" style="font-family: Calibri, sans-serif; font-size: 11pt;">https://techroads.org/docker-logging-to-the-local-os-that-works-with-compose-and-rsyslog/</a></p><p>However, any failed login attemps were logged with the local docker IP as follows:<br /><br /></p>
<pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="overflow-wrap: normal; word-wrap: normal;"> Nov 3 14:43:07 shiny2 docker-mongodb[18367]: {"t":{"$date":"2021-11-03T03:43:07.203+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn10","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"root","authenticationDatabase":"admin","client":"<b><span style="color: red;">172.18.0.2</span></b>:61065","result":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}
</code></pre>
You can see that the IP address of <b>172.18.0.2</b> is a docker generated IP address and there's no point of having fail2ban blocking 172.18.0.2 from further access to the VM when the originating IP address is something completely different.<div><br /></div><div>So how do we get docker to log the originating IP in the logs?</div><div><br /></div><h2 style="text-align: left;"><span style="color: #0b5394;">The solution</span></h2><div>There's a little known configuration you can use in the <b>docker-compose.yml </b>that can tell docker to share the same network space as the host rather than trying to virtualise the network environment, shown as follows:<br /><br /></div>
<pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: #f0f0f0; border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="overflow-wrap: normal; word-wrap: normal;"> version: "3.8"
services:
mongodb:
image : mongo
container_name: mongodb
command: [--auth]
environment:
- MONGODB_ROOT_PASSWORD=changeme
- MONGODB_USERNAME=someuser
- MONGODB_PASSWORD= changeme
- MONGODB_DATABASE=someDB
volumes:
- /mnt/srv/databases/mongodb:/data/db
ports:
- 27017:27017
restart: unless-stopped
# Share the same network space as host to preserve IPs in log files
<b><span style="color: red;">network_mode: host</span></b>
logging:
driver: syslog
options:
tag: docker-mongodb
networks:
default:
external: true
name: db-net
</code></pre>
I hope this helps someone save hours of searching like I did!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-61973927722488975732021-09-12T15:10:00.005-07:002021-09-12T15:10:51.923-07:00Apple iOS upload to SAMBA drive fails with "The operation couldn't be completed. Operation cancelled"<p> Apple iOS 14.5 and upwards (so far) has a introduced a bug preventing apple users from uploading or creating files on their local samba drives on the network. The upload will start but eventually fail with an error popup:</p><blockquote><p><span style="color: #990000; font-family: courier; font-size: medium;"><b style="background-color: #cccccc;">The operation couldn't be completed. Operation cancelled</b></span></p></blockquote><div style="text-align: left;">If you have access to the samb configuration file (smb.conf) then you can apply a known workaround. In my instance, my samba is hosted on an Ubuntu machine. Edit the file found at <b>/etc/samba/smb.conf</b> and under the <b>[global] </b>section add the following 2 lines:</div><div style="text-align: left;"><br /></div><div style="text-align: left;"><br /></div>
<pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: #f0f0f0; border: 1px dashed rgb(204, 204, 204); color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal; word-wrap: normal;"> [global]
vfs objects = fruit streams_xattr
fruit:nfs_aces = no
</code></pre>
<br /><div>Then restart samba</div><div><br /></div><div><pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: #f0f0f0; border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; width: 677.156px;"><code style="overflow-wrap: normal; word-wrap: normal;">service smbd restart</code></pre></div>
And you're good to go!Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8027951124477290753.post-48638045382630082582021-06-21T20:13:00.000-07:002021-06-21T20:15:14.883-07:00Springboot + Vue.js + Azure Active Directory Authentication<p class="MsoTitle"><span style="font-size: 24.0pt;">Springboot + VueJS + Azure AD
Authentication<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This is a configuration guide on how to setup a SpringBoot
application running as a backend API with a VueJS frontend, authenticated using
Microsoft’s Azure Active Directory (AD).<o:p></o:p></p>
<p class="MsoNormal"> In this case, we are using Azure only for authentication, whereby
staff and students of the university, can use their own credentials to access our
custom-built platforms.<o:p></o:p></p>
<p class="MsoNormal">The process by which authentication occurs step-by-step can
be summarised in the diagram below:<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-eFZpcQE6hf4/YNFR41GXg4I/AAAAAAAEEhU/YwwPAt5O1HUloSUNAIePIRKTTgGsq-eugCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="395" data-original-width="499" height="358" src="https://lh3.googleusercontent.com/-eFZpcQE6hf4/YNFR41GXg4I/AAAAAAAEEhU/YwwPAt5O1HUloSUNAIePIRKTTgGsq-eugCLcBGAsYHQ/w451-h358/image.png" width="451" /></a></div><br /><br />
<p class="MsoNormal">In the above diagram we can link the following:<o:p></o:p></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;"><b>Client:</b> VueJS client
application<o:p></o:p></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;"><b>Resource Server</b>: Springboot
API<o:p></o:p></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;"><b>Authorization server:</b> Azure
Active Directory<o:p></o:p></p>
<p class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;"><o:p> </o:p></p>
<p class="MsoNormal">This approach relies on Azure configuration in both the
front-end VueJS app and Springboot API app<o:p></o:p></p>
<p class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->1)<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><!--[endif]-->When a user first visits your secured VueJS
application, the user needs to be identified and is redirected to the Azure
authentication server for login. Once the user has been identified, the access
token can be requrested from the Authorization server (in this case Azure).<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->2)<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><!--[endif]-->The access token is stored internally in the
VueJS app and used later in subsequent calls to the API as part of the ‘Bearer’
header. The API (Resource server), accepts the token and forwards it to Azure
for validation of the access Token. <o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->3)<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><!--[endif]-->Azure responds to the API confirming the access
token is valid, including what roles the user has.<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->4)<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><!--[endif]-->The API checks the user roles have access to the
specific resource (API) being requested.<o:p></o:p></p>
<p class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]-->5)<span style="font-size: 7pt; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;">
</span><!--[endif]-->Once access has been confirmed, the API returns
the result to the VueJS client.<o:p></o:p></p>
<p class="MsoNormal">In this configuration, the API is no longer responsible for
generating tokens. All tokens are managed by Azure. The API simply passes
tokens around for validation.<o:p></o:p></p>
<h1>Azure<o:p></o:p></h1>
<p class="MsoNormal"><a href="https://portal.azure.com/">https://portal.azure.com</a>
<br />
<a href="https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade">https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade</a>
<o:p></o:p></p>
<p class="MsoNormal">Reference: <a href="https://devblogs.microsoft.com/azure-sdk/vue-js-user-authentication/">https://devblogs.microsoft.com/azure-sdk/vue-js-user-authentication/</a>
<o:p></o:p></p>
<p class="MsoNormal">Goto “Active Directory” </p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-D7EY0P6UuBE/YNFSOzsJp4I/AAAAAAAEEhk/MtdQwVH7gpAu3XFCv3W9huJTlyJkVaQ_gCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="73" data-original-width="72" height="54" src="https://lh3.googleusercontent.com/-D7EY0P6UuBE/YNFSOzsJp4I/AAAAAAAEEhk/MtdQwVH7gpAu3XFCv3W9huJTlyJkVaQ_gCLcBGAsYHQ/w54-h54/image.png" width="54" /></a></div><br /><o:p></o:p><p></p>
<p class="MsoNormal">Before we can configure our VueJS and Springboot app, we
need to create a new App in Azure. There are already many online instructions
on how to do this, so I will cut to the chase:<o:p></o:p></p>
<h3>Registration<o:p></o:p></h3>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-Dj4Kym3Hbik/YNFSVKvhruI/AAAAAAAEEho/c127f9E7aIoiiPWrGN0egljqE-1TixitACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="410" data-original-width="423" height="456" src="https://lh3.googleusercontent.com/-Dj4Kym3Hbik/YNFSVKvhruI/AAAAAAAEEho/c127f9E7aIoiiPWrGN0egljqE-1TixitACLcBGAsYHQ/w473-h456/image.png" width="473" /></a></div><br /><br />
<h3>App IDs<o:p></o:p></h3>
<p class="MsoNormal">Once the new App has been registered we need to take note of
the Client ID and Tenant ID:<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-iHVRo42aYFs/YNFSe-2OYXI/AAAAAAAEEhw/9xbjGZf68Z4vTBMa2la9jTYQF4_lN_stACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="256" data-original-width="588" height="242" src="https://lh3.googleusercontent.com/-iHVRo42aYFs/YNFSe-2OYXI/AAAAAAAEEhw/9xbjGZf68Z4vTBMa2la9jTYQF4_lN_stACLcBGAsYHQ/w557-h242/image.png" width="557" /></a></div><br /><br />
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h3>Redirect URIs<o:p></o:p></h3>
<p class="MsoNormal">Add the redirect URIs hosting your VueJS app.<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-t8rDNOP4ZlY/YNFSxQnLn8I/AAAAAAAEEiA/eeWzSJi4mdodP8P21_08dBcFAasJnuK4gCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="347" data-original-width="490" height="336" src="https://lh3.googleusercontent.com/-t8rDNOP4ZlY/YNFSxQnLn8I/AAAAAAAEEiA/eeWzSJi4mdodP8P21_08dBcFAasJnuK4gCLcBGAsYHQ/w474-h336/image.png" width="474" /></a></div><br /><br />
<p class="MsoNormal">The URIs should be for your VueJS app. In my case,
localhost:3001 was for VueJS.<o:p></o:p></p>
<h3>API Permissions<o:p></o:p></h3>
<p class="MsoNormal">Make sure the App has the following API permissions set:<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-dcb6EO6-enU/YNFS33uwx_I/AAAAAAAEEiE/rQOo8HBUFtYfuvv72Fz4x9QxyHCC6TsagCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="307" data-original-width="701" height="270" src="https://lh3.googleusercontent.com/-dcb6EO6-enU/YNFS33uwx_I/AAAAAAAEEiE/rQOo8HBUFtYfuvv72Fz4x9QxyHCC6TsagCLcBGAsYHQ/w617-h270/image.png" width="617" /></a><span style="text-align: left;"> </span></div>
<h3>App Roles<o:p></o:p></h3>
<p class="MsoNormal">For each app we can create our own roles, and assign users
to those roles.<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/--nsQWGAXm7k/YNFS--kODDI/AAAAAAAEEiI/Auu1GylaPOQozScwE8Mcxr33k4LZ0Y2ogCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="265" data-original-width="697" height="247" src="https://lh3.googleusercontent.com/--nsQWGAXm7k/YNFS--kODDI/AAAAAAAEEiI/Auu1GylaPOQozScwE8Mcxr33k4LZ0Y2ogCLcBGAsYHQ/w648-h247/image.png" width="648" /></a></div><br /><br />
<p class="MsoNormal">Here, we’ve created 2 roles with values ROLE_ADMIN and
ROLE_RESEARCHER. The allowed member types should be set to ‘Users/Groups’.<o:p></o:p></p>
<h3>Manifest (Might be optional)<o:p></o:p></h3>
<p class="MsoNormal">In the manifest file set the accessTokenAcceptedVersion to 2<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-CNuwzZAVbFE/YNFTHDPVwOI/AAAAAAAEEiQ/J-NzioBVlgwQmstaVEoO0bzavtJ0epvgACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="411" data-original-width="608" height="216" src="https://lh3.googleusercontent.com/-CNuwzZAVbFE/YNFTHDPVwOI/AAAAAAAEEiQ/J-NzioBVlgwQmstaVEoO0bzavtJ0epvgACLcBGAsYHQ/image.png" width="320" /></a></div>
<h3>Enterprise Application Mode<o:p></o:p></h3>
<p class="MsoNormal">Switch to Enterprise Application mode by searching for your
App<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-_-rry7wth2Q/YNFTKm5AHUI/AAAAAAAEEiY/EfJAyuXrmcQS6IW3pPvPU0nzXW9IwpOEgCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="334" data-original-width="483" height="324" src="https://lh3.googleusercontent.com/-_-rry7wth2Q/YNFTKm5AHUI/AAAAAAAEEiY/EfJAyuXrmcQS6IW3pPvPU0nzXW9IwpOEgCLcBGAsYHQ/w469-h324/image.png" width="469" /></a></div>
<h3>Add users and assign roles<o:p></o:p></h3>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-MyjEI46U8BI/YNFTr-ONk9I/AAAAAAAEEi0/dYulggZ_bw4_epI3HH5Thfa6CR1GvmBywCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="372" data-original-width="605" height="299" src="https://lh3.googleusercontent.com/-MyjEI46U8BI/YNFTr-ONk9I/AAAAAAAEEi0/dYulggZ_bw4_epI3HH5Thfa6CR1GvmBywCLcBGAsYHQ/w486-h299/image.png" width="486" /></a></div><br /><br />
<p class="MsoNormal"><o:p> </o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-3Hb12waMbXA/YNFTzYLX2HI/AAAAAAAEEi4/6jYDcOSLF4Y-7fDhrgtoR08Q2mGfc87XwCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="292" data-original-width="339" height="240" src="https://lh3.googleusercontent.com/-3Hb12waMbXA/YNFTzYLX2HI/AAAAAAAEEi4/6jYDcOSLF4Y-7fDhrgtoR08Q2mGfc87XwCLcBGAsYHQ/image.png" width="279" /></a></div><br /><br />
<p class="MsoNormal">Pick a user and pick a role, and save.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And that’s it for the Azure setup.<o:p></o:p></p>
<span style="font-family: "Calibri",sans-serif; font-size: 11.0pt; line-height: 107%; mso-ansi-language: EN-AU; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><br clear="all" style="mso-special-character: line-break; page-break-before: always;" />
</span>
<p class="MsoNormal"><o:p> </o:p></p>
<h1>VueJS + MSAL 2.0<o:p></o:p></h1>
<p class="MsoNormal">Reference: <a href="https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/lib/msal-browser">https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/lib/msal-browser</a><o:p></o:p></p>
<p class="MsoNormal">Reference: <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-spa-app-registration">https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-spa-app-registration</a>
<o:p></o:p></p>
<p class="MsoNormal">Reference: <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/id-tokens">https://docs.microsoft.com/en-us/azure/active-directory/develop/id-tokens</a>
<o:p></o:p></p>
<p class="MsoNormal">Microsoft has released a javascript library for developers
to authenticate users with Azure Active Directory called <b>MSAL.js</b><o:p></o:p></p>
<p class="MsoNormal">In your existing VueJS application, from the command line,
import the following libraries.<o:p></o:p></p>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<p class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;"><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial;">npm I @azure/core-http<o:p></o:p></span></p>
<p class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;"><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial;">npm I @azure/msal-browser</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">At the time of writing this document, the versions were:<o:p></o:p></p>
<p class="MsoNormal" style="background: #1E1E1E; line-height: 14.25pt; margin-bottom: .0001pt; margin-bottom: 0cm;"><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"> </span><span style="color: #9cdcfe; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">"@azure/core-http"</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">: </span><span style="color: #ce9178; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">"^1.2.6"</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">,<o:p></o:p></span></p>
<p class="MsoNormal" style="background: #1E1E1E; line-height: 14.25pt; margin-bottom: .0001pt; margin-bottom: 0cm;"><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"> </span><span style="color: #9cdcfe; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">"@azure/msal-browser"</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">: </span><span style="color: #ce9178; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">"^2.14.2"</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">,<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Under services folder, create a file called <b>auth-azure.service.js </b>from: <o:p></o:p></p>
<p class="MsoNormal"><a href="https://github.com/Philip-Wu/VueJS-MSAL/blob/main/auth-azure.service.js">https://github.com/Philip-Wu/VueJS-MSAL/blob/main/auth-azure.service.js</a><o:p></o:p></p>
<p class="MsoNormal">Modify the file to set the <b>clientId</b>, <b>tenantId</b> and <b>authority</b> url in the config. The
authority URL may be different depending on how your organization is setup. For
more information on authority: <a href="https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration#authority">https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration#authority</a>
<o:p></o:p></p>
<p class="MsoNormal">In my case the authority was configured as:<o:p></o:p></p>
<p class="MsoNormal" style="background: #1E1E1E; line-height: 14.25pt; margin-bottom: .0001pt; margin-bottom: 0cm;"><span style="color: #9cdcfe; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">authority:</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"> </span><span style="color: #ce9178; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">'https://login.microsoftonline.com/<tenantId></span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">,<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Where the URL is suffixed with the tenantId.<o:p></o:p></p>
<p class="MsoNormal">Please take note of the defaultScope used for acquiring tokens.
<o:p></o:p></p>
<p class="MsoNormal" style="background: #1E1E1E; line-height: 14.25pt; margin-bottom: .0001pt; margin-bottom: 0cm;"><span style="color: #6a9955; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">This is super important. Otherwise, we get an error about 'Invalid signature' due to receiving an access token in v1.0 format.</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"><o:p></o:p></span></p>
<p class="MsoNormal" style="background: #1E1E1E; line-height: 14.25pt; margin-bottom: .0001pt; margin-bottom: 0cm;"><span style="color: #6a9955; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">The backend API will attempt to validate the token using a 2.0 endpoint, which is not suited for an v1.0 access token. So to force</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">
</span><span style="color: #6a9955; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">v2.0 accessToken, we use the ./default. This was not mentioned in any formal documentation</span><span style="color: #d4d4d4; font-family: Consolas; font-size: 10.5pt; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The defaultScope should be <b><your_client_id>/.default </b>in order to get the v2.0 format for
access tokens. This should already be coded in the function from the github
file. This was not well documented by Azure.<o:p></o:p></p>
<p class="MsoNormal">The <b>.env</b> file should
contain the redirect URI that should be the same as what was configured in
Azure earlier:<o:p></o:p></p>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<p class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; padding: 0cm;">VUE_APP_AZURE_AUTH_REDIRECT_URI='http://localhost:3001/'<o:p></o:p></p>
</div>
<p class="MsoNormal">It must be prefixed with <b>VUE_APP_</b> in order for VueJS to make it available to the
application. Otherwise, it will be undefined.<o:p></o:p></p>
<p class="MsoNormal"><br /></p>
<p class="MsoNormal">In the <b>router/index.js</b>
file, when the user clicks on the login button, we can invoke the auth-azure script:<o:p></o:p></p>
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div> {</div><div> <span style="color: #9cdcfe;">path</span><span style="color: #9cdcfe;">:</span> <span style="color: #ce9178;">"/login"</span>,</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #9cdcfe;">:</span> <span style="color: #ce9178;">"login"</span>,</div><div> <span style="color: #9cdcfe;">component</span><span style="color: #9cdcfe;">:</span> <span style="color: #9cdcfe;">Login</span>,</div><div> <span style="color: #dcdcaa;">beforeEnter</span><span style="color: #9cdcfe;">:</span> <span style="color: #569cd6;">async</span> (<span style="color: #9cdcfe;">to</span>, <span style="color: #9cdcfe;">from</span>, <span style="color: #dcdcaa;">next</span>) <span style="color: #569cd6;">=></span> { </div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'Login route, user: '</span>+<span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">user</span>());</div><div> <span style="color: #c586c0;">if</span>(! <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">user</span>()) {</div><div> <span style="color: #c586c0;">if</span> (<span style="color: #9cdcfe;">app</span> != <span style="color: #569cd6;">undefined</span>) {</div><div> <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">appSignIn</span>();</div><div> } </div><div> } <span style="color: #c586c0;">else</span> {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'already authenticated user:'</span>);</div><div> }</div><br /><div> <span style="color: #dcdcaa;">next</span>();</div><div> }
},<span style="background-color: transparent;"> </span></div></div>
<p class="MsoNormal">Further down in the <b>router/index.js</b> file, we want to stored
a redirectPath in the session if a secured page was requested:<o:p></o:p></p>
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #4fc1ff;">router</span>.<span style="color: #dcdcaa;">beforeEach</span>((<span style="color: #9cdcfe;">to</span>, <span style="color: #9cdcfe;">from</span>, <span style="color: #dcdcaa;">next</span>) <span style="color: #569cd6;">=></span> {</div><div> <span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">publicPages</span> = [<span style="color: #ce9178;">'/login'</span>, <span style="color: #ce9178;">'/register'</span>, <span style="color: #ce9178;">'/'</span>,<span style="color: #ce9178;">'/home'</span>];</div><div> <span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">authRequired</span> = !<span style="color: #4fc1ff;">publicPages</span>.<span style="color: #dcdcaa;">includes</span>(<span style="color: #9cdcfe;">to</span>.<span style="color: #9cdcfe;">path</span>);</div><div> <span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">loggedIn</span> = <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">isLoggedIn</span>();</div><div> <span style="color: #6a9955;">// trying to access a restricted page + not logged in</span></div><div> <span style="color: #6a9955;">// redirect to login page</span></div><div> <span style="color: #c586c0;">if</span> (<span style="color: #4fc1ff;">authRequired</span> && !<span style="color: #4fc1ff;">loggedIn</span>) {</div><div> <span style="color: #9cdcfe;">sessionStorage</span>.<span style="color: #dcdcaa;">setItem</span>(<span style="color: #ce9178;">'redirectPath'</span>, <span style="color: #9cdcfe;">to</span>.<span style="color: #9cdcfe;">path</span>);</div><div> </div><div> <span style="color: #dcdcaa;">next</span>(<span style="color: #ce9178;">'/login'</span>);</div><div> } <span style="color: #c586c0;">else</span> {</div><div> <span style="color: #dcdcaa;">next</span>();</div><div> }</div><div>});</div><br /></div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In <b>main.js</b>, we
handle the created() event by checking if the user is already logged in:<o:p></o:p></p>
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">var</span> <span style="color: #9cdcfe;">app</span> = <span style="color: #569cd6;">new</span> <span style="color: #4ec9b0;">Vue</span>({</div><div> <span style="color: #9cdcfe;">router</span>,</div><div> <span style="color: #9cdcfe;">store</span>,</div><div> <span style="color: #dcdcaa;">created</span>() {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'app created'</span>)</div><div> <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">init</span>()</div><div> }, </div><div> <span style="color: #569cd6;">async</span> <span style="color: #dcdcaa;">mounted</span>() {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'app mounted'</span>)</div><br /><div> <span style="color: #569cd6;">let</span> <span style="color: #9cdcfe;">redirectPath</span> = <span style="color: #9cdcfe;">sessionStorage</span>.<span style="color: #dcdcaa;">getItem</span>(<span style="color: #ce9178;">'redirectPath'</span>); </div><div> <span style="color: #c586c0;">if</span> (<span style="color: #9cdcfe;">authAzure</span>.<span style="color: #dcdcaa;">isLoggedIn</span>() && <span style="color: #9cdcfe;">redirectPath</span>) {</div><br /><div> <span style="color: #6a9955;">// Define function to be used as callback</span></div><div> <span style="color: #569cd6;">let</span> <span style="color: #dcdcaa;">redirectFunc</span> = <span style="color: #569cd6;">function</span>() {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'redirecting to '</span>+ <span style="color: #9cdcfe;">redirectPath</span>)</div><div> <span style="color: #9cdcfe;">sessionStorage</span>.<span style="color: #dcdcaa;">removeItem</span>(<span style="color: #ce9178;">'redirectPath'</span>)</div><div> <span style="color: #4fc1ff;">router</span>.<span style="color: #dcdcaa;">push</span>(<span style="color: #9cdcfe;">redirectPath</span>) </div><div> }</div><br /><div> <span style="color: #c586c0;">if</span> (! <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #9cdcfe;">waitingOnAccessToken</span>) {</div><div> <span style="color: #dcdcaa;">redirectFunc</span>()</div><div> } <span style="color: #c586c0;">else</span> {</div><div> <span style="color: #6a9955;">// Register callback to be executed when accessToken has been assigned</span></div><div> <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #9cdcfe;">accessTokenCallbacks</span>.<span style="color: #dcdcaa;">push</span>(<span style="color: #dcdcaa;">redirectFunc</span>)</div><div> }</div><div> }</div><br /><div> },</div><div> <span style="color: #dcdcaa;">render</span><span style="color: #9cdcfe;">:</span> <span style="color: #dcdcaa;">h</span> <span style="color: #569cd6;">=></span> <span style="color: #dcdcaa;">h</span>(<span style="color: #9cdcfe;">App</span>),</div><div> <span style="color: #dcdcaa;">renderError</span>(<span style="color: #dcdcaa;">h</span>, <span style="color: #9cdcfe;">err</span>) {</div><div> <span style="color: #c586c0;">return</span> <span style="color: #dcdcaa;">h</span>(<span style="color: #ce9178;">'pre'</span>, { <span style="color: #9cdcfe;">style</span><span style="color: #9cdcfe;">:</span> { <span style="color: #9cdcfe;">color</span><span style="color: #9cdcfe;">:</span> <span style="color: #ce9178;">'red'</span> }}, <span style="color: #9cdcfe;">err</span>.<span style="color: #9cdcfe;">stack</span>)</div><div> } </div><div> });</div></div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Create a file called <b>auth-header.js</b>
that will be used as a function for supplying the Bearer Authorization header
with the acquired accessToken:<o:p></o:p></p>
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #569cd6;">function</span> <span style="color: #dcdcaa;">authHeaderAzure</span>() {</div><div> <span style="color: #569cd6;">let</span> <span style="color: #9cdcfe;">accessToken</span> = <span style="color: #9cdcfe;">authAzure</span>.<span style="color: #9cdcfe;">accessToken</span>;</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #9cdcfe;">app</span>+<span style="color: #ce9178;">'using token: '</span>+<span style="color: #9cdcfe;">accessToken</span>);</div><br /><div> <span style="color: #c586c0;">if</span> (<span style="color: #9cdcfe;">accessToken</span>) {</div><div> <span style="color: #c586c0;">return</span> { <span style="color: #9cdcfe;">Authorization</span><span style="color: #9cdcfe;">:</span> <span style="color: #ce9178;">'Bearer '</span> + <span style="color: #9cdcfe;">accessToken</span> };</div><div> } <span style="color: #c586c0;">else</span> {</div><div> <span style="color: #c586c0;">return</span> {};</div><div> }</div><br /><div>}</div></div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Create a file <b>auth.module.js
</b>to handle the dispatches from auth-azure.service.js. If the user was
attempting to access a secured page directly, then they are redirected the
requested page immediately after authentication.<o:p></o:p></p>
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #c586c0;">import</span> <span style="color: #9cdcfe;">authAzure</span> <span style="color: #c586c0;">from</span> <span style="color: #ce9178;">'../services/auth-azure.service'</span>;</div><div><span style="color: #c586c0;">import</span> <span style="color: #9cdcfe;">app</span> <span style="color: #c586c0;">from</span> <span style="color: #ce9178;">'../main'</span></div><br /><div><span style="color: #6a9955;">/**</span></div><div><span style="color: #6a9955;"> * Used for rendering the navigation componenet, Nav.vue, to manage the state of being Logged in vs Logged out.</span></div><div><span style="color: #6a9955;"> * Using the Vuex.store we can trigger Nav.vue to re-render upon logging in or out.</span></div><div><span style="color: #6a9955;"> */</span></div><br /><div><span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">user</span> = <span style="color: #9cdcfe;">authAzure</span>?.<span style="color: #dcdcaa;">user</span>()</div><br /><div><span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">initialState</span> = <span style="color: #4fc1ff;">user</span></div><div> ? { <span style="color: #9cdcfe;">loggedIn</span><span style="color: #9cdcfe;">:</span> <span style="color: #569cd6;">true</span> }</div><div> : { <span style="color: #9cdcfe;">loggedIn</span><span style="color: #9cdcfe;">:</span> <span style="color: #569cd6;">false</span> };</div><br /><div><span style="color: #c586c0;">export</span> <span style="color: #569cd6;">const</span> <span style="color: #4fc1ff;">auth</span> = {</div><div> <span style="color: #9cdcfe;">namespaced</span><span style="color: #9cdcfe;">:</span> <span style="color: #569cd6;">true</span>,</div><div> <span style="color: #9cdcfe;">state</span><span style="color: #9cdcfe;">:</span> <span style="color: #4fc1ff;">initialState</span>,</div><br /><div> <span style="color: #9cdcfe;">mutations</span><span style="color: #9cdcfe;">:</span> {</div><div> <span style="color: #dcdcaa;">loginSuccess</span>(<span style="color: #9cdcfe;">state</span>) {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'mutation loginSuccess'</span>)</div><div> <span style="color: #9cdcfe;">state</span>.<span style="color: #9cdcfe;">loggedIn</span> = <span style="color: #569cd6;">true</span>; </div><div> },</div><div> <span style="color: #dcdcaa;">loginFailure</span>(<span style="color: #9cdcfe;">state</span>) {</div><div> <span style="color: #9cdcfe;">state</span>.<span style="color: #9cdcfe;">loggedIn</span> = <span style="color: #569cd6;">false</span>;</div><div> },</div><div> <span style="color: #dcdcaa;">logout</span>(<span style="color: #9cdcfe;">state</span>) {</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'mutation logoutSuccess'</span>)</div><div> <span style="color: #9cdcfe;">state</span>.<span style="color: #9cdcfe;">loggedIn</span> = <span style="color: #569cd6;">false</span>;</div><div> },</div><div> </div><div> },</div><br /><div> <span style="color: #9cdcfe;">actions</span><span style="color: #9cdcfe;">:</span> {</div><div> <span style="color: #dcdcaa;">loginSuccess</span>({<span style="color: #9cdcfe;">commit</span>}) {</div><div> <span style="color: #dcdcaa;">commit</span>(<span style="color: #ce9178;">'loginSuccess'</span>)</div><br /><div> <span style="color: #6a9955;">// Redirect user if a page was loaded directly in the browser</span></div><div> <span style="color: #569cd6;">let</span> <span style="color: #9cdcfe;">redirectPath</span> = <span style="color: #9cdcfe;">sessionStorage</span>.<span style="color: #dcdcaa;">getItem</span>(<span style="color: #ce9178;">'redirectPath'</span>);</div><div> <span style="color: #9cdcfe;">console</span>.<span style="color: #dcdcaa;">log</span>(<span style="color: #ce9178;">'redirectPath: '</span>+<span style="color: #9cdcfe;">redirectPath</span>)</div><div> <span style="color: #c586c0;">if</span> (<span style="color: #9cdcfe;">redirectPath</span>) {</div><div> <span style="color: #9cdcfe;">sessionStorage</span>.<span style="color: #dcdcaa;">removeItem</span>(<span style="color: #ce9178;">'redirectPath'</span>);</div><div> <span style="color: #9cdcfe;">app</span>.<span style="color: #9cdcfe;">$router</span>.<span style="color: #dcdcaa;">push</span>(<span style="color: #9cdcfe;">redirectPath</span>)</div><div> }</div><br /><div> },</div><div> <span style="color: #dcdcaa;">logout</span>({<span style="color: #9cdcfe;">commit</span>}) {</div><div> <span style="color: #dcdcaa;">commit</span>(<span style="color: #ce9178;">'logout'</span>)</div><div> },</div><div> <span style="color: #dcdcaa;">loginFailure</span>({<span style="color: #9cdcfe;">commit</span>}) {</div><div> <span style="color: #dcdcaa;">commit</span>(<span style="color: #ce9178;">'loginFailure'</span>)</div><div> }</div><div> }</div><div>};</div></div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<h2>Troubleshooting<o:p></o:p></h2>
<p class="MsoNormal">Can use the following tool to decode an access token. Take
carefully notice of the token version number:<o:p></o:p></p>
<p class="MsoNormal"><a href="https://jwt.ms/">https://jwt.ms/</a><o:p></o:p></p>
<span style="font-family: "Calibri",sans-serif; font-size: 11.0pt; line-height: 107%; mso-ansi-language: EN-AU; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><br clear="all" style="mso-special-character: line-break; page-break-before: always;" />
</span>
<p class="MsoNormal"><o:p> </o:p></p>
<h1>Springboot + Azure authentication<o:p></o:p></h1>
<p class="MsoNormal">Reference: <a href="https://docs.microsoft.com/en-us/java/api/overview/azure/spring-boot-starter-active-directory-readme?view=azure-java-stable">https://docs.microsoft.com/en-us/java/api/overview/azure/spring-boot-starter-active-directory-readme?view=azure-java-stable</a>
(accessing a Resource server section)<o:p></o:p></p>
<p class="MsoNormal">Reference: <a href="https://github.com/Azure/azure-sdk-for-java/tree/azure-spring-boot-starter-active-directory_3.5.0/sdk/spring/azure-spring-boot-samples/azure-spring-boot-sample-active-directory-resource-server">https://github.com/Azure/azure-sdk-for-java/tree/azure-spring-boot-starter-active-directory_3.5.0/sdk/spring/azure-spring-boot-samples/azure-spring-boot-sample-active-directory-resource-server</a><o:p></o:p></p>
<p class="MsoNormal">Reference: <a href="https://developer.okta.com/blog/2019/06/20/spring-preauthorize">https://developer.okta.com/blog/2019/06/20/spring-preauthorize</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">To setup our Springboot app as a Resource server, we add the
following Azure libraries to our dependency to our <b>pom.xml</b> file:<o:p></o:p></p>
<p class="MsoNormal" style="background: #2B2B2B; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;"><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"><dependency><br />
<groupId></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">com.azure.spring</span><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"></groupId><br />
<artifactId></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">azure-spring-boot-starter-active-directory</span><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"></artifactId><br />
<version></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">3.5.0</span><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"></version><br />
</dependency><br />
<dependency><br />
<groupId></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">org.springframework.boot</span><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"></groupId><br />
<artifactId></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">spring-boot-starter-oauth2-resource-server</span><span style="color: #e8bf6a; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"></artifactId><br />
</dependency></span><span style="color: #a9b7c6; font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Then we need to configure our springboot security by
creating the following file:<o:p></o:p></p>
<pre style="background: #2B2B2B;"><span style="color: #bbb529; font-size: 9.0pt;">@Slf4j<br />
@Configuration<br />
@EnableWebSecurity<br />
@EnableGlobalMethodSecurity</span><span style="color: #a9b7c6; font-size: 9.0pt;">(</span><span style="color: #d0d0ff; font-size: 9.0pt;">prePostEnabled </span><span style="color: #a9b7c6; font-size: 9.0pt;">= </span><span style="color: #cc7832; font-size: 9.0pt;">true</span><span style="color: #a9b7c6; font-size: 9.0pt;">)<br />
</span><span style="color: #cc7832; font-size: 9.0pt;">class </span><span style="color: #a9b7c6; font-size: 9.0pt;">WebSecurityConfig </span><span style="color: #cc7832; font-size: 9.0pt;">extends </span><span style="color: #a9b7c6; font-size: 9.0pt;">AADResourceServerWebSecurityConfigurerAdapter{<br />
</span><span style="color: grey; font-size: 9.0pt;"><br />
</span><span style="color: #bbb529; font-size: 9.0pt;">@Autowired<br />
</span><span style="color: #a9b7c6; font-size: 9.0pt;">UserDetailsServiceImpl </span><span style="color: #9876aa; font-size: 9.0pt;">userDetailsService<br />
<br />
</span><span style="color: #bbb529; font-size: 9.0pt;">@Autowired<br />
</span><span style="color: #cc7832; font-size: 9.0pt;">private </span><span style="color: #a9b7c6; font-size: 9.0pt;">AuthEntryPointJwt </span><span style="color: #9876aa; font-size: 9.0pt;">unauthorizedHandler<br />
<br />
</span><span style="color: #bbb529; font-size: 9.0pt;">@Override<br />
</span><span style="color: #cc7832; font-size: 9.0pt;">protected void </span><span style="color: #a9b7c6; font-size: 9.0pt;">configure(HttpSecurity http) </span><span style="color: #cc7832; font-size: 9.0pt;">throws </span><span style="color: #a9b7c6; font-size: 9.0pt;">Exception{<br />
</span><span style="color: #cc7832; font-size: 9.0pt;">super</span><span style="color: #a9b7c6; font-size: 9.0pt;">.configure(http)<br />
</span><span style="color: grey; font-size: 9.0pt;"><br />
</span><span style="color: #a9b7c6; font-size: 9.0pt;">.antMatchers(</span><span style="color: #6a8759; font-size: 9.0pt;">"/"</span><span style="color: #a9b7c6; font-size: 9.0pt;">,</span><span style="color: #6a8759; font-size: 9.0pt;">"/v1/var/**"</span><span style="color: #a9b7c6; font-size: 9.0pt;">,</span><span style="color: #6a8759; font-size: 9.0pt;">"/home"</span><span style="color: #a9b7c6; font-size: 9.0pt;">,</span><span style="color: #6a8759; font-size: 9.0pt;">"/signin"</span><span style="color: #a9b7c6; font-size: 9.0pt;">,</span><span style="color: #6a8759; font-size: 9.0pt;">"/login"</span><span style="color: #a9b7c6; font-size: 9.0pt;">,</span><span style="color: #6a8759; font-size: 9.0pt;">"/hash"</span><span style="color: #a9b7c6; font-size: 9.0pt;">, </span><span style="color: #6a8759; font-size: 9.0pt;">"/signup"</span><span style="color: #a9b7c6; font-size: 9.0pt;">, </span><span style="color: #6a8759; font-size: 9.0pt;">"/explorer/**"</span><span style="color: #a9b7c6; font-size: 9.0pt;">).permitAll()</span><span style="color: grey; font-size: 9.0pt;"><br />
</span><span style="color: #a9b7c6; font-size: 9.0pt;">.anyRequest().authenticated()
<span> </span>}
}
</span>}} </pre>
<p class="MsoNormal">Next we need to configure our Azure app by providing the
tenant-id and client-id in the <b>application.yml</b>
file:<o:p></o:p></p>
<pre style="background: #2B2B2B;"><b><span style="color: #cc7832; font-size: 9.0pt;">azure</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">:<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">activedirectory</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">:<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">tenant-id</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">: <azure app tentant id><br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">client-id</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">: <azure app client id><o:p></o:p></span></pre>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">If you want to debug an azure related error, I would also
highly suggest setting the root logging level to DEBUG in the application.yml.
This helped me to resolve a misleading error message:<o:p></o:p></p>
<pre style="background: #2B2B2B;"><b><span style="color: #cc7832; font-size: 9.0pt;">logging</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">:<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">file</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">:<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">path</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">: logs<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">level</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">:<br />
</span><b><span style="color: #cc7832; font-size: 9.0pt;">root</span></b><span style="color: #a9b7c6; font-size: 9.0pt;">: DEBUG<o:p></o:p></span></pre>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Before we can define the role access for our API in the
controller, we need to work out how the <b>roles
are renamed by Azure (or the library)</b>. This is where I spent most of my
time as none of this was mentioned in any of the official documentation or any
forums. This is where the logging level of DEBUG was handy. <o:p></o:p></p>
<p class="MsoNormal">If the roles do not match exactly between Azure and the
Springboot API, then on the browser we may see network errors when debugging
using the “Developer tools” of the browser:<o:p></o:p></p>
<div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-Sp803Ua-5_c/YNFUymUo4CI/AAAAAAAEEjE/cH6KKnbyK70v00ojFjAxLZHJ3TrxJQF1QCLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="370" data-original-width="697" height="335" src="https://lh3.googleusercontent.com/-Sp803Ua-5_c/YNFUymUo4CI/AAAAAAAEEjE/cH6KKnbyK70v00ojFjAxLZHJ3TrxJQF1QCLcBGAsYHQ/w631-h335/image.png" width="631" /></a></div><br /><br />
<p class="MsoNormal">Here we can see the error says “<b>insufficient_scope</b>” and “<b>The
request requires higher privileges than provided by the access token</b>”. For me
this was misleading, suggesting that the issue was related to the Azure API
Permissions, but that was not the case. The API permission ‘User.Read’, should
be sufficient privileges for authentication and accessing our API. Rather than
an API permission issue, it was really a <b>ROLE
configuration</b> issue.<o:p></o:p></p>
<p class="MsoNormal">On the API server, we can see the following logs:<o:p></o:p></p>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<p class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; padding: 0cm;">2021-06-21 11:42:38,284
{HH:mm:ss.SSS} [http-nio-8081-exec-1] DEBUG
o.s.s.o.s.r.w.BearerTokenAuthenticationFilter - Set SecurityContextHolder to
BearerTokenAuthentication
[Principal=com.azure.spring.aad.webapi.AADOAuth2AuthenticatedPrincipal@4581efe6,
Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails
[RemoteIpAddress=0:0:0:0:0:0:0:1, SessionId=null], Granted
Authorities=[SCOPE_User.Read, <b><span style="color: red;">APPROLE_ROLE_ADMIN</span></b>]]<o:p></o:p></p>
</div>
<p class="MsoNormal">We can see that Azure is prefixing my configured roles with <b>APPROLE_<o:p></o:p></b></p>
<p class="MsoNormal">This means, in my controllers, I need to specify my role
access using APPROLE_ROLE_ADMIN as follows:<o:p></o:p></p>
<pre style="background: #2B2B2B;"><span style="color: #bbb529; font-size: 9.0pt;">@Slf4j</span><span style="color: #a9b7c6; font-size: 9.0pt;"><br />
</span><span style="color: #bbb529; font-size: 9.0pt;">@RestController<br />
@RequestMapping</span><span style="color: #a9b7c6; font-size: 9.0pt;">(</span><span style="color: #d0d0ff; font-size: 9.0pt;">path</span><span style="color: #a9b7c6; font-size: 9.0pt;">=</span><span style="color: #6a8759; font-size: 9.0pt;">"/v1/var"</span><span style="color: #a9b7c6; font-size: 9.0pt;">)<br />
</span><span style="color: #bbb529; font-size: 9.0pt;">@PreAuthorize</span><span style="color: #a9b7c6; font-size: 9.0pt;">(</span><span style="color: #6a8759; font-size: 9.0pt;">"hasAnyAuthority('APPROLE_ROLE_ADMIN', 'APPROLE_ROLE_RESEARCHER')"</span><span style="color: #a9b7c6; font-size: 9.0pt;">)<br />
</span><span style="color: #cc7832; font-size: 9.0pt;">class </span><span style="color: #a9b7c6; font-size: 9.0pt;">VariantAnnotationController {</span>…</pre>
<p class="MsoNormal">Here we use the <b>@PreAuthorize</b>
annotation and the <b>hasAnyAuthority</b>
method to control access to the API based on the Azure roles previously
configured.<o:p></o:p></p>
<span style="font-family: "Calibri",sans-serif; font-size: 11.0pt; line-height: 107%; mso-ansi-language: EN-AU; mso-ascii-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-language: AR-SA; mso-bidi-theme-font: minor-bidi; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;"><br clear="all" style="mso-special-character: line-break; page-break-before: always;" />
</span>
<p class="MsoNormal"><o:p> </o:p></p>
<h1>Other notes:<o:p></o:p></h1>
<p class="MsoNormal">Initially I had tried to use the VueJS plugin as a wrapper
for the MSAL.js client library, but I had trouble getting that to work, perhaps
because it was initially designed for an older version of the MSAL library and
hasn’t been updated for a while. Just for reference the vueJs plugin is the ‘vue-msal’
plugin: <a href="https://github.com/mvertopoulos/vue-msal">https://github.com/mvertopoulos/vue-msal</a>
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p> </p>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8027951124477290753.post-48742556317565296582021-05-20T21:20:00.002-07:002021-05-20T21:20:12.932-07:00"cannot read property prototype of undefined" lb4 repository decorator<p> Using Loopback4, I was trying to create new repositories and noticed afterwards when trying to start the app, I had the following error:</p>
<pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal; word-wrap: normal;"> TypeError: Cannot read property 'prototype' of undefined
at Object.repository (C:\sts-workspace\rodentity-nodejs\node_modules\@loopback\repository\src\decorators\repository.decorator.ts:145:53)
at Object.<anonymous> (C:\sts-workspace\rodentity-nodejs\src\services\unfinished-genotyping-plates.service.ts:17:6)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object.<anonymous> (C:\sts-workspace\rodentity-nodejs\src\services\index.ts:3:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
</code></pre>
<br />
<div>The error message is not very helpful. But after debugging it, I worked I had a missed a step when creating the repository. I needed to add a line to the<b> application.ts</b> file to register the newly created repositories as follows:</div><div><br /></div>
<pre style="background-attachment: initial; background-clip: initial; background-color: #f0f0f0; background-image: URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif); background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; background: #f0f0f0; border: 1px dashed rgb(204, 204, 204); color: black; font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal; word-wrap: normal;"> this.repository(MouseRepository);
</code></pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-24612265962574648822020-11-15T18:10:00.001-08:002020-11-15T18:10:42.790-08:00Loopback LB4, mysql SSL configuration<p> I had a tough time trying to find documentation on how to configure loopback 4 to use SSL certs for communicating with a mySQL backend. So here's how I did it:</p><p><br /></p><p><br /></p>
<pre style="font-family:arial;font-size:12px;border:1px dashed #CCCCCC;width:99%;height:auto;overflow:auto;background:#f0f0f0;;background-image:URL(http://2.bp.blogspot.com/_z5ltvMQPaa8/SjJXr_U2YBI/AAAAAAAAAAM/46OqEP32CJ8/s320/codebg.gif);padding:0px;color:#000000;text-align:left;line-height:20px;"><code style="color:#000000;word-wrap:normal;"> dataSource: {
name: "mysql",
connector: "mysql",
// url: process.env.DS_URL, // don't use URL because SSL doesn't work with it
host: process.env.DB_HOST,
port: 3306,
ssl: {
rejectUnauthorized: false,
ca: fs.readFileSync(process.env.mysqlSSLCAcert, { encoding: 'utf8', flag: 'r' }),
key: fs.readFileSync(process.env.mysqlSSLClientKey, { encoding: 'utf8', flag: 'r' }), //client
cert: fs.readFileSync(process.env.mysqlSSLClientCert, { encoding: 'utf8', flag: 'r' }), // client
},
user: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD,
database: "facility"
},
</code></pre>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-51975960680113162022020-05-28T21:47:00.001-07:002020-05-28T22:28:49.252-07:00RabbitMQ proxied through apache httpd web server<h2>
Problem</h2>
RabbitMQ wasn't allowing us to add exchanges through the web console. Clicking on the button resulted in the PUT request being sent to apache, but apache wasn't forwarding the traffic onto rabbitMQ.<br />
<br />
This was confirmed by checking the apache logs. Indeed the request was received, but in the rabbitMQ logs, there was no change in activity. This would a suggest a problem at the apache level.<br />
<br />
At the browser level, debugging shows a 404 status code was returned, which suggests that apache wasn't able to find handler for the request.<br />
<h2>
Solution</h2>
<br />
Reference: <a href="https://serverfault.com/questions/639327/rabbitmq-behind-apache-mod-proxy-not-resolving-deep-link">https://serverfault.com/questions/639327/rabbitmq-behind-apache-mod-proxy-not-resolving-deep-link</a><br />
<br />
Modify your httpd.conf or ssl.conf to have the following<br />
<pre style="border-radius: 3px; border: 0px; box-sizing: inherit; color: #242729; font-family: Consolas, Menlo, Monaco, "Lucida Console", "Liberation Mono", "DejaVu Sans Mono", "Bitstream Vera Sans Mono", "Courier New", monospace, sans-serif; font-size: 13px; font-stretch: inherit; font-variant-east-asian: inherit; font-variant-numeric: inherit; line-height: inherit; margin-bottom: 1em; max-height: 600px; overflow-wrap: normal; overflow: auto; padding: 12px 8px; vertical-align: baseline; width: auto;"><div style="box-sizing: border-box; font-family: "Segoe UI", system-ui, "Apple Color Emoji", "Segoe UI Emoji", sans-serif; font-size: 14px; white-space: normal;">
<div>
Edit /etc/rabbitmq/rabbitmq.conf and add</div>
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> management.path_prefix = /mq
</code></pre>
<pre></pre>
<pre></pre>
<div>
</div>
<div>
Edit /etc/httpd/conf.d/ssl.conf and add the following rules:</div>
</div>
</pre>
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="overflow-wrap: normal;"> <b><span style="color: red;"> AllowEncodedSlashes NoDecode</span></b>
ProxyPass /mq http://localhost:15672/mq <b><span style="color: red;">nocanon </span></b>
ProxyPassReverse /mq http://localhost:15672/mq
</code></pre>
<br />
It's important to note that a <b><span style="color: red;">trailing slash is required</span></b> when entering the URL path in the browser. For example:<br />
<br />
https://myhost/mq<span style="color: red;"><b>/</b></span><br />
<span style="color: red;"><b><br /></b></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-36417886031004972282020-02-14T13:09:00.002-08:002020-02-15T02:38:42.150-08:00How to remove bloatware from Oppo Reno Z<div>
Its unbelievable how much bloatware apps are installed by defualt on Oppo phones. But there is a way to remove them using some simple command-line tools.</div>
<div>
<br /></div>
This article describes how to remove the bloatware installed by default on a new Oppo Reno Z using the adb tool.<br />
<div>
<br /></div>
<div>
Follow this guide to install ADB: </div>
<div>
<br /></div>
<div>
<a href="https://www.xda-developers.com/uninstall-carrier-oem-bloatware-without-root-access/" target="_blank">https://www.xda-developers.com/uninstall-carrier-oem-bloatware-without-root-access/</a></div>
<div>
<br /></div>
<div>
To install ADB go here:</div>
<div>
<br /></div>
<div>
<a href="https://developer.android.com/studio/releases/platform-tools.html">https://developer.android.com/studio/releases/platform-tools.html</a></div>
<div>
<br /></div>
<div>
In Windows 10 using powershell, launch ADB as follows</div>
<div>
<br /></div>
<blockquote class="tr_bq">
cmd /c adb shell</blockquote>
<br />
Most of the bloatware is installed under the package "com.coloros"<br />
<br />
<blockquote class="tr_bq">
pm list packages | grep coloros</blockquote>
<br />
Here are the commands I used for removing the bloatware apps:<br />
<br />
pm uninstall -k --user 0 com.coloros.gamespace<br />
pm uninstall -k --user 0 com.coloros.weather.service<br />
pm uninstall -k --user 0 com.coloros.weather2<br />
pm uninstall -k --user 0 com.coloros.gallergy3d<br />
pm uninstall -k --user 0 com.coloros.musiclink<br />
pm uninstall -k --user 0 com.coloros.phonemanager<br />
pm uninstall -k --user 0 com.coloros.calculator<br />
pm uninstall -k --user 0 com.coloros.screenrecorder<br />
pm uninstall -k --user 0 com.coloros.alarmclock<br />
pm uninstall -k --user 0 com.coloros.compass2<br />
pm uninstall -k --user 0 com.coloros.oppomultiapp<br />
pm uninstall -k --user 0 com.coloros.soundrecorder<br />
pm uninstall -k --user 0 com.coloros.oppopods<br />
pm uninstall -k --user 0 com.coloros.onekeylockscreen<br />
pm uninstall -k --user 0 com.coloros.video<br />
pm uninstall -k --user 0 com.coloros.<br />
<br />
<br />
That cleaned up quite a bit and removed icons from the screen.<br />
<br />
Unfortunately, there were some apps I couldn't remove such as the Oppo Contacts, and Clone Phone.<br />
I'd be interested to hear if anybody figured out how to remove them, or other bloatware found on their phone.<br />
<br />
<h2>
Restore uninstalled apps</h2>
The apps are not really deleted, they're just hidden. To view a list of all packages including the uninstalled apps use<br />
<br />
<blockquote class="tr_bq">
pm list packages -u | grep coloros</blockquote>
<br />
To restore an app in adb shell<br />
<br />
<blockquote class="tr_bq">
cmd package install-existing com.coloros.usbselection</blockquote>
<br />
<br />
Good luck<br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-64009278685719592652019-09-12T20:29:00.003-07:002019-09-12T20:29:47.620-07:00Redis + Java + Redisson = Encoding issuesUsing Reddison/Java to save to Redis, I noticed when using the redis-cli command-line client, that the saved records were encoded funny, as follows:<br />
<br />
<br />
127.0.0.1:6379> hgetall SessionMap<br />
1) "\xfc(gB2muKkpNJVgDhiXQA5fGELNcFkFcBszLh5wqcXX"<br />
2) "\xfc\x0214"<br />
3) "\xfc(lh9wm3gpV767f6KrvZfVnHO5gKKUFMHfL1KJEsRC"<br />
4) "\xfc\x0214"<br />
<br />
Following from their online documentation, in Java code, I had the following:<br />
<br />
<blockquote class="tr_bq">
<pre style="font-family: "courier new"; font-size: 9pt;"><span style="background-color: #eeeeee;">RMap map = redisson.getMap("SessionMap");</span></pre>
</blockquote>
Turns out we need to specify the encoding of the string by using this bit of code instead:<br />
<br />
<blockquote class="tr_bq" style="font-family: "Courier New"; font-size: 9pt;">
<span style="background-color: #cccccc;">RMap map = redisson.getMap("SessionMap", <b><span style="color: red;">new StringCodec("UTF-8")</span></b>);</span></blockquote>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-37702553485084764822019-06-30T17:18:00.000-07:002019-06-30T17:18:00.313-07:00Docker Containers Missing!For many months we've been trying to track down the cause for the sudden termination of our docker containers.<br />
<br />
When running the "<b>docker container ls</b>" command, nothing is shown. It almost looks like they're missing.<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
</code></pre>
<br />
However running the process command "<b>docker ps -a</b>" does indeed indicate that our containers were exited:<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@patient-archive-cpi centos]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4a9b66fdd624 aehrc/ontoserver:ctsa-5.2 "/run.sh run" 7 weeks ago Exited (143) 2 days ago ontoserver
9e8314f52de2 postgres "docker-entrypoint.s…" 7 weeks ago Exited (0) 2 days ago 5432/tcp docker_db_1
</code></pre>
<br />
<br />
The command for viewing "<b>dockers logs <container_id></container_id></b>", indicated that the container was being shutdown. But there were no errors or reason for the shutdown.<br />
<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> 2019-06-28 03:41:48.302Z INFO 1 --- [ Thread-4] s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@63d4e2ba: startup date [Fri May 10 04:11:42 GMT 2019]; root of context hierarchy
2019-06-28 03:41:48.313Z INFO 1 --- [ Thread-4] o.s.jmx.export.annotation.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
2019-06-28 03:41:48.314Z INFO 1 --- [ Thread-4] o.s.jmx.export.annotation.AnnotationMBeanExporter : Unregistering JMX-exposed beans
2019-06-28 03:41:48.316Z INFO 1 --- [ Thread-4] o.s.scheduling.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'batchRunner'
2019-06-28 03:41:48.319Z INFO 1 --- [ Thread-4] o.s.scheduling.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'jobRunner'
2019-06-28 03:41:48.319Z INFO 1 --- [ Thread-4] o.s.scheduling.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'auditReportJobRunner'
2019-06-28 03:41:48.329Z INFO 1 --- [ Thread-4] o.s.orm.jpa.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2019-06-28 03:41:48.331Z INFO 1 --- [ Thread-4] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2019-06-28 03:41:48.343Z INFO 1 --- [ Thread-4] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
</code></pre>
<br />
Running the docker command for viewing events showed nothing of interest<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> docker events --filter container=ontoserver --since '2019-06-27'
</code></pre>
<br />
After some digging around it turns out that there was an automated yum update to docker in the <b>/var/log/yum.log</b><br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> Jun 28 03:41:44 Updated: docker-ce-cli.x86_64 1:18.09.7-3.el7
Jun 28 03:41:48 Updated: containerd.io.x86_64 1.2.6-3.3.el7
Jun 28 03:41:52 Updated: docker-ce.x86_64 3:18.09.7-3.el7
</code></pre>
<br />
<div style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin-top: 12px; overflow-wrap: break-word; padding: 0px;">
Turns out in docker there's a feature for re-enabling the docker containers after a docker update called '<b>live-restore</b>'</div>
<div style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin-top: 12px; overflow-wrap: break-word; padding: 0px;">
Added a file <b>/etc/docker/daemon.json</b> with the following contents</div>
<div class="codehilite" style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin: 10px 0px 0px; padding: 0px;">
<pre style="background: rgb(244, 245, 247); border-radius: 3px; border: 0px; font-family: SFMono-Medium, "SF Mono", "Segoe UI Mono", "Roboto Mono", "Ubuntu Mono", Menlo, Courier, monospace; font-size: 12px; letter-spacing: 0px; line-height: 1.4; overflow-wrap: normal; overflow-x: auto; padding: 5px 10px;">{
"live-restore": true
}
</pre>
</div>
<div style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin-top: 12px; overflow-wrap: break-word; padding: 0px;">
Then restarted docker</div>
<blockquote style="background-color: white; border-bottom: none; border-image: initial; border-left: 1px solid rgb(204, 204, 204); border-right: none; border-top: none; color: #707070; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin: 10px 0px 0px; padding: 10px 30px;">
<div style="display: inline-block; overflow-wrap: break-word; padding: 0px;">
systemctl reload docker</div>
</blockquote>
<div style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin-top: 12px; overflow-wrap: break-word; padding: 0px;">
</div>
<div style="background-color: white; color: #172b4d; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin-top: 12px; overflow-wrap: break-word; padding: 0px;">
Reference: <a data-is-external-link="true" href="https://docs.docker.com/config/containers/live-restore/" rel="nofollow" style="color: #0052cc; text-decoration-line: none;">https://docs.docker.com/config/containers/live-restore/</a></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-90224852539863919142019-04-07T23:55:00.003-07:002019-04-07T23:55:31.547-07:00flutter Android dependency 'androidx.core:core' has different version for the compile (1.0.0) and runtime (1.0.1) classpath<div>
In the android/build.gradle file set the gradle version to 3.3.2 as follows:</div>
<div>
<br /></div>
<div>
<pre style="background-color: #2b2b2b; color: #a9b7c6; font-family: 'Courier New'; font-size: 9.0pt;">dependencies {
classpath <span style="color: #6a8759;">'com.android.tools.build:gradle:3.3.2'</span><span style="color: #6a8759;">
</span>}</pre>
</div>
<div>
<br /></div>
<div>
<br /></div>
Reference: <a href="https://github.com/flutter/flutter/issues/27254">https://github.com/flutter/flutter/issues/27254</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-67548278206507271482019-04-07T19:42:00.002-07:002019-04-07T19:42:33.840-07:00flutter "detected problems with api compatibility"In the android/app/build.gradle file change the targetSdkVersion from 27 to 28.<br />
<br />
Reference: <a href="https://stackoverflow.com/questions/49957255/espresso-test-on-android-p-preview-detected-problems-with-api-compatibility-err">https://stackoverflow.com/questions/49957255/espresso-test-on-android-p-preview-detected-problems-with-api-compatibility-err</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-16795761329406176952019-04-05T20:19:00.002-07:002019-04-05T20:19:20.584-07:00flutter "unable to start activity" "ensureInitializationComplete must be called after startInitialization"Flutter app fails to launch with the following error:<br />
<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> Caused by: java.lang.IllegalStateException: ensureInitializationComplete must be called after startInitialization
at io.flutter.view.FlutterMain.ensureInitializationComplete(FlutterMain.java:190)
at io.flutter.app.FlutterActivityDelegate.onCreate(FlutterActivityDelegate.java:156)
at io.flutter.app.FlutterActivity.onCreate(FlutterActivity.java:89)
at net.pawpalflutter.MainActivity.onCreate(MainActivity.java:10)
at android.app.Activity.performCreate(Activity.java:6672)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1140)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2612)
... 9 more
</code></pre>
<br />
<br />
This error occurred when loading app from Android Studio to a mobile phone. It seems the transfer of the app may have been interrupted abruptly and didn't close properly, causing all subsequent transfers to fail.<br />
<br />
<h2>
The Solution</h2>
Power off and on the phone<br />
<br />
Reference: <a href="https://github.com/flutter/flutter/issues/14513">https://github.com/flutter/flutter/issues/14513</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-86687964705448227042019-03-31T04:43:00.002-07:002019-03-31T04:43:28.912-07:00Intellij Dart Analysis Server Bad State: Too many elements<div>
This error was also causing an issue "<b>cannot find declaration to go to</b>"</div>
After doing a flutter upgrade, I had to do the following:<div>
<br /></div>
<div>
<ol>
<li>Close IntelliJ</li>
<li>delete the project's .idea folder and any *.iml files</li>
<li>Reopen the project in intellij</li>
</ol>
<div>
If that doesn't work try File -> Invalidate caches / restart and repeat the steps above</div>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-48615722828253113792019-03-03T14:52:00.001-08:002019-03-03T14:52:13.034-08:00PlatformException(sign_in_failed, com.google.android.gms.common.api.ApiException: 8: null)Using the flutter framework for android development, I received the following error during google signin:<br />
<br />
<blockquote class="tr_bq">
PlatformException(sign_in_failed, com.google.android.gms.common.api.ApiException: 8: null)</blockquote>
<br />
While it wasn't obvious from the error message what the problem was, it turns out there was no network connection available. So if you get this error, check that you have access to the internet!!<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-2380310099437907772019-02-19T14:13:00.003-08:002019-02-19T14:13:44.108-08:00Node.js Centos 7 production configuration<br />
<h2>
<span lang="EN-US">Install nodejs</span></h2>
<div class="MsoNormal">
<span lang="EN-US">Download<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; line-height: 107%; mso-bidi-font-family: "Lucida Console";">wget --no-check-certificate
https://nodejs.org/dist/v10.15.1/node-v10.15.1-linux-x64.tar.xz</span><span lang="EN-US"><o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<span lang="EN-US">Move the downloaded file /usr/local<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Then unpack<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; line-height: 107%; mso-bidi-font-family: "Lucida Console";">tar xf node-v10.15.1-linux-x64.tar.xz</span><span lang="EN-US"><o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<span lang="EN-US">Add node executables to the path by adding
symlinks to /usr/bin<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">ln -s /usr/local/node/bin/node /usr/bin/node<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">ln -s /usr/local/node/bin/npm /usr/bin/npm<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">ln -s /usr/local/node/bin/npx /usr/bin/npx<o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<br /></div>
<h2>
<span lang="EN-US"><o:p>Install pm2</o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">To run node as service we need to install a
nodejs package<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; line-height: 107%; mso-bidi-font-family: "Lucida Console";">npm install pm2@latest –g<o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<span lang="EN-US">We need to run set the path to pm2<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">ln -s /usr/local/node/bin/pm2 /usr/bin/pm2<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US" style="font-family: "Lucida Console"; font-size: 9.0pt; line-height: 107%; mso-bidi-font-family: "Lucida Console";">ln -s /usr/local/node/bin/pm2
/usr/bin/pm2-runtime</span><span lang="EN-US"><o:p></o:p></span></div>
</div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: 16.5pt;">
<br /></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: 16.5pt;">
For security reasons we should run our app NOT as root. We create a new user nodejs to run our app</div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span lang="EN-US" style="color: #3a3a3a; font-family: "Courier New"; font-size: 10.5pt; mso-fareast-font-family: "Times New Roman";">useradd nodejs<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span lang="EN-US" style="color: #3a3a3a; font-family: "Courier New"; font-size: 10.5pt; mso-fareast-font-family: "Times New Roman";">chown -R
nodejs:nodejs /usr/local/node<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span lang="EN-US" style="color: #3a3a3a; font-family: "Courier New"; font-size: 10.5pt; mso-fareast-font-family: "Times New Roman";">chown –R
nodejs:nodejs /usr/local/</span><span style="color: #6060ff; font-family: "Lucida Console"; font-size: 9.0pt; mso-ansi-language: EN-AU; mso-bidi-font-family: "Lucida Console";">node-v10.15.1-linux-x64<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="color: #6060ff; font-family: "Lucida Console"; font-size: 9.0pt; mso-ansi-language: EN-AU; mso-bidi-font-family: "Lucida Console";">#
as root</span><span lang="EN-US" style="color: #3a3a3a; font-family: "Courier New"; font-size: 10.5pt; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span lang="EN-US" style="color: #3a3a3a; font-family: "Courier New"; font-size: 10.5pt; mso-fareast-font-family: "Times New Roman";">pm2
startup system –u nodejs –-hp /home/nodjs<o:p></o:p></span></div>
</div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">To start the application as the nodejs user, switch to the user nodejs<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">su nodejs<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">pm2 start /usr/local/node/apps/chat/server.js<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">pm2 list<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US"># to automatically start the app on reboots</span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">pm2 save</span></div>
</div>
<div class="MsoNormal">
<span lang="EN-US"><o:p><br /></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><o:p>You should see that the app is running as user nodejs </o:p></span></div>
<div class="MsoNormal">
<br /><span lang="EN-US"><o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-T54i5tf6DQ4/XGx-fRtaSbI/AAAAAAADGJE/saV4P8eLXSQs2w6QZNHLZntZoqhcHwzEACLcBGAs/s1600/pm2-list.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="78" data-original-width="761" height="64" src="https://4.bp.blogspot.com/-T54i5tf6DQ4/XGx-fRtaSbI/AAAAAAADGJE/saV4P8eLXSQs2w6QZNHLZntZoqhcHwzEACLcBGAs/s640/pm2-list.png" width="640" /></a></div>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US">Likewise to stop<o:p></o:p></span></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">#other
commands<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">pm2 stop /usr/local/node/apps/chat/server.js<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span lang="EN-US">pm2
delete /usr/local/node/apps/chat/server.js<o:p></o:p></span></div>
</div>
<br />Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-8027951124477290753.post-87683224261946799952019-02-11T13:45:00.002-08:002019-02-11T13:45:49.732-08:00Install ArangoDB on Centos 7<br />
<div class="MsoNormal">
The instructions on the official arangodb website do not
work, so I’m documenting the steps I took so that it may help others.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The first problem is adding arangodb as a repo due to
invalid SSL certificates. To get around the issue:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div style="background: whitesmoke; border: solid #E5E4E4 1.0pt; mso-border-alt: solid #E5E4E4 .75pt; mso-element: para-border-div; padding: 7.0pt 7.0pt 7.0pt 7.0pt;">
<div class="MsoNormal" style="background: whitesmoke; border: none; line-height: normal; margin-bottom: 7.5pt; mso-border-alt: solid #E5E4E4 .75pt; mso-padding-alt: 7.0pt 7.0pt 7.0pt 7.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; word-break: break-all;">
<span style="color: #333333; font-family: Consolas; font-size: 10.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">cd /etc/yum.repos.d/<o:p></o:p></span></div>
<div class="MsoNormal" style="background: whitesmoke; border: none; line-height: normal; margin-bottom: 7.5pt; mso-border-alt: solid #E5E4E4 .75pt; mso-padding-alt: 7.0pt 7.0pt 7.0pt 7.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; word-break: break-all;">
<span style="color: #333333; font-family: Consolas; font-size: 10.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">curl -OLk <a href="https://download.arangodb.com/arangodb34/RPM/arangodb.repo">https://download.arangodb.com/arangodb34/RPM/arangodb.repo</a><o:p></o:p></span></div>
<div class="MsoNormal" style="background: whitesmoke; border: none; line-height: normal; margin-bottom: 7.5pt; mso-border-alt: solid #E5E4E4 .75pt; mso-padding-alt: 7.0pt 7.0pt 7.0pt 7.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt; word-break: break-all;">
<br /></div>
</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The added ‘k’ parameter connects to the repo insecurely
skipping the step of verifying the certificates.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Before we can run the yum command to install arangodb, we
need to configure the arangodb.repo to skip SSL checks by adding the following
to the file /etc/yum.repos.d/arangodb.repo<o:p></o:p></div>
<div style="background: #DBE5F1; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DBE5F1; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
sslverify=0<o:p></o:p></div>
</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Then run the command to install:<o:p></o:p></div>
<div style="background: #DBE5F1; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DBE5F1; border: none; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
yum install arangodb3<o:p></o:p></div>
</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
References:<o:p></o:p></div>
<div class="MsoNormal">
<a href="https://superuser.com/questions/1057905/adding-no-check-certificate-to-yum-repositories">https://superuser.com/questions/1057905/adding-no-check-certificate-to-yum-repositories</a><o:p></o:p></div>
<div class="MsoNormal">
<a href="https://www.arangodb.com/download-major/centos/">https://www.arangodb.com/download-major/centos/</a><o:p></o:p></div>
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-31138892758692415072018-12-10T18:04:00.000-08:002018-12-10T18:04:06.392-08:00Grails 2.4.4 Spring tool suite STS - failed to read artifact descriptor Recently, we started having issues downloading dependencies for our Grails projects running on Java 7 with the following error:<div>
<br /></div>
<blockquote class="tr_bq">
<span style="font-size: large;"><b> failed to read artifact descriptor </b></span></blockquote>
I was basically trying to run a 'grails clean' command on the project through STS eclipse (version 3.6.4). I was aware with the community finally disabling support for version TLSv1 and suspected it might be related. I made all attempts to specify TLSv1.1 and TLSv1.2 in various configurations in STS without any success. I tried in the STS eclipse INI file using the argument<br />
<br />
<blockquote class="tr_bq">
-Dhttps.protocols=TLSv1.1,TLSv1.2</blockquote>
I also tried setting the https.protocol in the JRE definition of eclipse:<br />
<br />
Windows -> Preferences -> Java -> Installed JREs -> jdk1.7.0_79 -> Edit -> Default VM arguments<br />
<br />
No luck either.<br />
<br />
I finally decided to run 'grails command' on the command-line where I had JDK version 8 running by default:<br />
<br />
<br />
<blockquote class="tr_bq">
C:\grails\grails-2.4.4\bin\grails clean</blockquote>
| JVM Version: 1.8.0_171<br />
| Application cleaned.<br />
<br />
You can see that using Java 8, it was able to download all the dependencies successfully.<br />
<br />
It's not ideal to have to use the command-line to get the dependencies downloaded, but once it's all cached locally, then you can switch back to STS to build and run grails-apps.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-38818253674085983552018-11-25T18:34:00.000-08:002018-11-25T18:34:16.189-08:00Ignoring OSSEC rulesTo ignore some errors in OSSEC we can configure our custom rules in <b>/var/ossec/rules/local_rules.xml</b><br />
<br />
In this case I'm going to ignore some Shibboleth errors I received in an email:<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="overflow-wrap: normal;"> OSSEC HIDS Notification.
2018 Nov 26 12:56:27
Received From: apn-lsrv01->/etc/httpd/logs/ssl_access_log
<b><span style="color: red;">Rule: 31122</span></b> fired (level 5) -> "Web server 500 error code (Internal Error)."
Src IP: 150.203.1.1
Portion of the log(s):
<b><span style="color: red;">150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937</span></b>
--END OF NOTIFICATION
</code></pre>
<br />
I've highlighted the relevant parts we'll need in red font above.<br />
<br />
Before we add new rules to ignore this error, we need to identify which <b>group </b>it belongs to.<br />
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="overflow-wrap: normal;"> cd /var/ossec/rules
grep -lir <b><span style="color: red;">31122 </span></b>.
./<span style="color: red;">web_rules.xml </span>
</code></pre>
<br />
Here we can see that the rule 31122 exists in the file web_rules.xml. Therefore the group that the rule belongs to is '<b>web</b>'<br />
<br />
Now let's analyze how ossec will decode the log error using tool called <b>ossec-logtest.</b><br />
Start ossec-logtest, run the command: /var/ossec/bin/ossec-logtest<br />
Then copy and paste the portion of the log you received in the email, and you should get a response similar to this:<br />
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal;"> [root@apn-lsrv01 bin]# .</code><code style="overflow-wrap: normal;"><span style="color: red;"><b>/ossec-logtest</b></span></code><code style="color: black; overflow-wrap: normal;">
2018/11/26 13:08:06 ossec-testrule: INFO: Reading local decoder file.
2018/11/26 13:08:06 ossec-testrule: INFO: Started (pid: 8696).
ossec-testrule: Type one log per line.
</code><code style="overflow-wrap: normal;"><span style="color: red;">150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937</span></code><code style="color: black; overflow-wrap: normal;">
**Phase 1: Completed pre-decoding.
full event: '150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937'
hostname: 'apn-lsrv01'
program_name: '(null)'
log: '150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937'
**Phase 2: Completed decoding.
decoder: 'web-accesslog'
srcip: '150.203.25.3'
srcuser: '-'
action: 'GET'
</code><code style="overflow-wrap: normal;"><span style="color: red;"><b>url: '/Shibboleth.sso/NIM/Artifact'</b></span></code><code style="color: black; overflow-wrap: normal;">
id: '500'
**Phase 3: Completed filtering (rules).
Rule id: '31122'
Level: '5'
Description: 'Web server 500 error code (Internal Error).'
**Alert to be generated.
</code></pre>
<br />
Here we can see that OSSEC decoded the log error with a <b>url </b>as <b style="background-color: #f0f0f0; color: red; font-family: arial; font-size: 12px;">'/Shibboleth.sso/NIM/Artifact'</b><br />
<b style="background-color: #f0f0f0; color: red; font-family: arial; font-size: 12px;"><br /></b>
This means when we write our rule to ignore this error, we need to specify the rule using a URL.<br />
<br />
Now we can proceed to create our rule by editing the /var/ossec/rules/local_rules.xml by adding the following to the end of the file:<br />
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal;"> <group name="</code><code style="overflow-wrap: normal;"><span style="color: red;">web</span></code><code style="color: black; overflow-wrap: normal;">," >
<rule id="100032" level="0">
<if_sid></code><code style="overflow-wrap: normal;"><span style="color: red;">31122</span></code><code style="color: black; overflow-wrap: normal;"></if_sid>
<url></code><code style="overflow-wrap: normal;"><span style="color: red;">/Shibboleth.sso</span></code><code style="color: black; overflow-wrap: normal;"></url>
<description>Ignore Shibboleth</description>
</rule>
</group>
</code></pre>
<br />
<br />
<ul>
<li>In this rule we specified that the rule belongs to group called 'web'.</li>
<li>The rule ID to which we are processing has ID 31122.</li>
<li>And the URL should start with /Shibboleth.sso</li>
</ul>
<br />
We can rerun our ossec-logtest without having to restart OSSEC.<br />
Now if we rerun ossec-logtest we should see the following:<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; overflow-wrap: normal;"> [root@apn-lsrv01 bin]# ./ossec-logtest
2018/11/26 13:11:17 ossec-testrule: INFO: Reading local decoder file.
2018/11/26 13:11:17 ossec-testrule: INFO: Started (pid: 9181).
ossec-testrule: Type one log per line.
150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937
**Phase 1: Completed pre-decoding.
full event: '150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937'
hostname: 'apn-lsrv01'
program_name: '(null)'
log: '150.203.25.3 - - [26/Nov/2018:12:56:25 +1100] "GET /Shibboleth.sso/NIM/Artifact HTTP/1.1" 500 937'
**Phase 2: Completed decoding.
decoder: 'web-accesslog'
srcip: '150.203.25.3'
srcuser: '-'
action: 'GET'
url: '/Shibboleth.sso/NIM/Artifact'
id: '500'
**Phase 3: Completed filtering (rules).
Rule id: '100032'
Level: '0'
Description: '</code><code style="overflow-wrap: normal;"><span style="color: red;"><b>Ignore Shibboleth</b></span></code><code style="color: black; overflow-wrap: normal;">'
</code></pre>
<br />
After all that testing , we are now ready to release our changes by restarting OSSEC.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-80869852406299906662018-10-31T04:44:00.000-07:002018-10-31T04:44:04.523-07:00Exetel vs Myrepublic NBN speed test Canberra<div>
Time 5:30pm</div>
<div>
<br /></div>
Exetel <div>
<br /></div>
<div>
Download: 45.5 Mbps</div>
<div>
Upload: 15 Mbps</div>
<div>
<br /></div>
<div>
MyRepublic</div>
<div>
<br /></div>
<div>
Download: 40 Mbps</div>
<div>
Upload: 8 Mbps</div>
<div>
<br /></div>
<div>
Exetel is faster, cheaper and has better phone plans and provides static ip for free. Exetel wins hands down.</div>
<div>
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-27796099443854201462018-08-28T20:49:00.000-07:002018-08-28T20:49:51.659-07:00OSSEC postfix email using localhost doesn't workOSSEC had issues sending me emails with the following error message in the /var/ossec/logs/ossec.log<br />
<br />
<b>ERROR: Error Sending email to localhost (smtp server)</b><br />
<br />
OSSEC was configured to use POSTFIX as my SMTP host as configured in the <b>/var/ossec/etc/ossec-server.conf</b><br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> <global>
<email_notification>yes</email_notification>
<email_to>philip.wu@anu.edu.au</email_to>
<smtp_server>localhost</smtp_server>
<email_from>patient-lookup@130.56.244.180</email_from>
</global>
</code></pre>
<br />
<br />
Once I changed localhost to 127.0.0.1, postfix emails worked:<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="word-wrap: normal;"> <global>
<email_notification>yes</email_notification>
<email_to>philip.wu@anu.edu.au</email_to>
<smtp_server><b><span style="color: red;">127.0.0.1</span></b></smtp_server>
<email_from>patient-lookup@130.56.244.180</email_from>
</global>
</code></pre>
<h3>
Reference:</h3>
<a href="https://github.com/ossec/ossec-hids/issues/1122" target="_blank">https://github.com/ossec/ossec-hids/issues/1122 </a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-7361429541145384802018-08-21T16:19:00.003-07:002018-08-21T16:19:48.990-07:00Encrypting postgres backupsLately I've been dabbling in the world of security. While I'd more interested doing other things like building features and tackling research problems, security is something that should be part of every day thinking when designing solutions. One area of security focuses on databases.<br />
<br />
While I've made the effort to doubly encrypt the postgres data at rest: One at the table column level, where certain fields are encrypted and two, at the file system level as a separate attached volume where postgres lives, these efforts would be useless if the database backups were stored as plain text. True, the encrypted fields would remain encrypted, but for peace of mind, let's encrypt the backups themselves!<br />
<br />
Here I'll using GPG (<span style="background-color: white; color: #545454; font-family: arial, sans-serif; font-size: x-small;">GNU Privacy Guard)</span> encryption on a Centos 7 machine with a postgres database. While there is a lot of information about GPG on the web, I couldn't find a comprehensive article on how to do this. So here we go!<br />
<br />
First let's install GPG<br />
<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-margin-bottom-alt: auto; mso-margin-top-alt: auto; mso-outline-level: 1; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
yum install gnupg2<o:p></o:p></div>
</div>
<br /><br />
Since I'm using the postgres user to perform the automated backups with ident authentication, we need to switch to the postgres user (assuming we are already the root user):<br />
<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">#
become the postgres user<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">su
postgres<o:p></o:p></span></div>
</div>
<br /><br />
When generating GPG keys, it will ask for a passphrase using TTY. Unfortunately, GPG doesn't work well when running the terminal in an 'su session' just as we have done with the above command. To workaround this, we issue the following command:<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">#
workaround to generate gpg key in a su session as postgres<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">script
/dev/null<o:p></o:p></span></div>
</div>
<br />
<span style="background-color: white; color: #242729; font-size: 15px;"><span style="font-family: inherit;">Redirecting the script to /dev/null causes screen to not try to write to the controlling terminal, so it doesn't hit the permission problem.</span></span><br />
<br />
Now can generate the GPG keys for the postgres user. You will be asked for a passphrase - keep this some where safe.<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">bash-4.2$
<b>gpg2 --gen-key</b><o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">gpg
(GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">This
is free software: you are free to change and redistribute it.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">There
is NO WARRANTY, to the extent permitted by law.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<br /></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Please
select what kind of key you want:<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> (1) RSA and RSA (default)<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> (2) DSA and Elgamal<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> (3) DSA (sign only)<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> (4) RSA (sign only)<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Your
selection?<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">RSA
keys may be between 1024 and 4096 bits long.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">What
keysize do you want? (2048)<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Requested
keysize is 2048 bits<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Please
specify how long the key should be valid.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> 0 = key does not expire<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> <n>
= key expires in n days<o:p></o:p></n></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> <n>w = key expires in n weeks<o:p></o:p></n></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> <n>m = key expires in n months<o:p></o:p></n></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> <n>y = key expires in n years<o:p></o:p></n></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Key
is valid for? (0)<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Key
does not expire at all<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Is
this correct? (y/N) y<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<br /></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">GnuPG
needs to construct a user ID to identify your key.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<br /></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"><span style="color: red;">Real
name: <b>postgres</b></span><o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Email
address:<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Comment:<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">You
selected this USER-ID:<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"> "postgres"<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<br /></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">Change
(N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">You
need a Passphrase to protect your secret key.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<br /></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">We
need to generate a lot of random bytes. It is a good idea to perform<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">some
other action (type on the keyboard, move the mouse, utilize the<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">disks)
during the prime generation; this gives the random number<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">generator
a better chance to gain enough entropy.<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">We
need to generate a lot of random bytes. It is a good idea to perform<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">some
other action (type on the keyboard, move the mouse, utilize the<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">disks)
during the prime generation; this gives the random number<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">generator
a better chance to gain enough entropy.<o:p></o:p></span></div>
</div>
<br />
The important thing to take note of is the 'Real name' which I've specified as 'postgres'. We will use this 'Real name' later when we perform the encryption.<br />
<br />
At this stage, it seemed to just hang without any idea if it was doing anything at all. In my first attempt, I had let it sit for over an hour and still nothing. Turns out Entropy takes a long time if there's no system activity. So let's introduce 'random activity' in another terminal:<br />
<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-outline-level: 1; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="background: #EFF0F1; color: #242729; font-family: Consolas; font-size: 10.0pt;">yum install
rng-tools<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-outline-level: 1; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">rngd
-r /dev/urandom<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-outline-level: 1; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";"><br /></span></div>
</div>
<br /><br />
After running the rngd command, you notice almost immediately in the other terminal, that the GPG key gen has complated. Now you can kill the rngd process that's still running in the background.<br />
<br />
<br />
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9pt;">ps -aux | grep rngd</span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">root<span style="mso-spacerun: yes;"> </span>25652<span style="mso-spacerun: yes;">
</span>0.0<span style="mso-spacerun: yes;"> </span>0.0<span style="mso-spacerun: yes;"> </span>13216<span style="mso-spacerun: yes;">
</span>368 ?<span style="mso-spacerun: yes;"> </span>Ss<span style="mso-spacerun: yes;"> </span>14:37<span style="mso-spacerun: yes;">
</span>0:00 <span style="color: #ff4040;">rngd</span> -r /dev/urandom<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">root<span style="mso-spacerun: yes;"> </span>25665<span style="mso-spacerun: yes;">
</span>0.0<span style="mso-spacerun: yes;"> </span>0.0 112704<span style="mso-spacerun: yes;"> </span>976 pts/0<span style="mso-spacerun: yes;">
</span>S+<span style="mso-spacerun: yes;"> </span>14:37<span style="mso-spacerun: yes;"> </span>0:00 grep --color=auto <span style="color: #ff4040;">rngd<o:p></o:p></span></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-layout-grid-align: none; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; text-autospace: none;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">kill -9 25652<o:p></o:p></span></div>
</div>
<br /><br />
<div>
To troubleshoot entropy availability, you can monitor entropy availability here which should sit at around 1450 when idle. When being consumed, it should be much lower:</div>
<div>
<br /></div>
<div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; line-height: 107%; mso-bidi-font-family: "Lucida Console";">watch
cat /proc/sys/kernel/random/entropy_avail</span><o:p></o:p></div>
</div>
</div>
<div>
<br /></div>
<div>
Now that we have our GPG keys, we are ready to encrypt files. Here I've created a script to execute the postgres backups, compression and encryption all in one step:</div>
<div>
<br /></div>
<div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">pg_dump
-U postgres db_name | gzip > /backups/db_backup_$(date
+%Y-%m-%d).psql.gz<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">gpg
-e –r <b>postgres </b>/home/backups/patient_lookup_$(date +%Y-%m-%d).psql.gz<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">rm
-rf /home/backups/patient_lookup_$(date +%Y-%m-%d).psql.gz<o:p></o:p></span></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
<span style="font-family: "Lucida Console"; font-size: 9.0pt; mso-bidi-font-family: "Lucida Console";">chmod
0600 -R /backups/*.gpg</span><span style="background: #EFF0F1; color: #242729; font-family: Consolas; font-size: 10.0pt;"><o:p></o:p></span></div>
</div>
</div>
<div>
<br /></div>
<div>
The first line using pg_dump generates a compressed GZ backup file.</div>
<div>
The second line then takes the GZ file and encrypts it, creating a new GPG file. The -e argument tells GPG to encrypt and the -r argument specifies the recipient which in this case is the postgres user that we specified earlier when generating the GPG keys.</div>
<div>
Since GPG creates a new file, we remove the GZ file in the third line.</div>
<div>
Then we only allow read/write permissions for the postgres user on the fourth line.</div>
<div>
<br /></div>
<div>
You can run the script on a cron job to routinely do your backups.</div>
<div>
<br /></div>
<div>
Of course, before you put this into production, you should check to ensure you can successfully decrypt the backups.</div>
<div>
<br /></div>
<div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
su postgres<o:p></o:p></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
script /dev/null<o:p></o:p></div>
<div class="MsoNormal" style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; border: none; line-height: normal; margin-bottom: 0.0001pt; padding: 0cm;">
gpg postgres_backup.gpg<o:p></o:p></div>
</div>
</div>
<div>
<br /></div>
<div>
If this helped you please like! Thx</div>
<h3>
References:</h3>
<div>
<div class="MsoNormal">
<a href="https://yanhan.github.io/posts/2017-09-27-how-to-use-gpg-to-encrypt-stuff.html">https://yanhan.github.io/posts/2017-09-27-how-to-use-gpg-to-encrypt-stuff.html</a><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<a href="https://www.digitalocean.com/community/tutorials/how-to-use-gpg-to-encrypt-and-sign-messages">https://www.digitalocean.com/community/tutorials/how-to-use-gpg-to-encrypt-and-sign-messages</a><o:p></o:p></div>
</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-73829989002738750032018-07-22T16:51:00.000-07:002018-07-22T16:51:42.794-07:00SELINX and postgres troubles<b>OS version:</b> Centos 7<br />
<br />
Upon enabling SELINUX, I noticed that the postgres service hadn't started. When I checked the logs I noticed the following error message:<br />
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@webserver data]# systemctl status postgresql.service
● postgresql.service - PostgreSQL database server
Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Sun 2018-07-22 23:18:52 UTC; 8s ago
Process: 2903 ExecStart=/usr/bin/pg_ctl start -D ${PGDATA} -s -o -p ${PGPORT} -w -t 300 (code=exited, status=1/FAILURE)
Process: 2897 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Jul 22 23:18:51 webserver.novalocal systemd[1]: Starting PostgreSQL database server...
</code><code style="word-wrap: normal;"><span style="color: red;"><b>Jul 22 23:18:51 webserver.novalocal pg_ctl[2903]: postgres cannot access the server configuration file "/var/lib/pgsql/data/postgresql.conf": Permission denied </b></span></code><code style="color: black; word-wrap: normal;">
Jul 22 23:18:52 webserver.novalocal pg_ctl[2903]: pg_ctl: could not start server
Jul 22 23:18:52 webserver.novalocal systemd[1]: postgresql.service: control process exited, code=exited status=1
Jul 22 23:18:52 webserver.novalocal systemd[1]: Failed to start PostgreSQL database server.
Jul 22 23:18:52 webserver.novalocal systemd[1]: Unit postgresql.service entered failed state.
Jul 22 23:18:52 webserver.novalocal systemd[1]: postgresql.service failed.
</code></pre>
<br />
To view the SELinux security context:<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@webserver var]# ls -Z /var/lib/pgsql/data/
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 base
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 global
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_clog
-rw-------. postgres postgres system_u:object_r:unlabeled_t:s0 pg_hba.conf
-rw-------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_ident.conf
drwx------. postgres postgres unconfined_u:object_r:postgresql_log_t:s0 pg_log
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_multixact
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_notify
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_serial
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_snapshots
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_stat_tmp
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_subtrans
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_tblspc
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_twophase
-rw-------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 PG_VERSION
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_xlog
</code><code style="word-wrap: normal;"><span style="color: red;"><b> -rw-------. postgres postgres system_u:object_r:default_t:s0 postgresql.conf</b></span></code><code style="color: black; word-wrap: normal;">
-rw-------. postgres postgres system_u:object_r:postgresql_db_t:s0 postmaster.opts
</code></pre>
<br />
We can see that the postgresql.conf file was incorrectly assigned a type of <b>default_t.</b><br />
<b><br /></b>
I noticed there were several other files in the postgresql data folder that had a similar problem. To fix the type for all files under the data folder run the following command:<br />
<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> chcon -R system_u:object_r:postgresql_db_t:s0 /var/lib/pgsql/data/**
</code></pre>
<br />
Rechecking the SElinux contexts:<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="word-wrap: normal;"> [root@webserver var]# ls -Z /var/lib/pgsql/data/
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 base
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 global
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_clog
-rw-------. postgres postgres system_u:object_r:unlabeled_t:s0 pg_hba.conf
-rw-------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_ident.conf
drwx------. postgres postgres unconfined_u:object_r:postgresql_log_t:s0 pg_log
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_multixact
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_notify
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_serial
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_snapshots
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_stat_tmp
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_subtrans
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_tblspc
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_twophase
-rw-------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 PG_VERSION
drwx------. postgres postgres unconfined_u:object_r:postgresql_db_t:s0 pg_xlog
<b><span style="color: red;">-rw-------. postgres postgres system_u:object_r:postgresql_db_t:s0 postgresql.conf</span></b>
-rw-------. postgres postgres system_u:object_r:postgresql_db_t:s0 postmaster.opts
</code></pre>
<br />
Now that it's fixed, turn on postgresql<br />
<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> service postgresql start
</code></pre>
<br />
<h3>
References:</h3>
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-working_with_selinux-selinux_contexts_labeling_filesUnknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-38860442736360163802018-05-24T15:00:00.002-07:002018-05-24T15:00:30.695-07:00Gradle OutOfMemoryError<br />
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<br /></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b style="mso-bidi-font-weight: normal;">OutOfMemoryError</b>
– It’s possible that gradle is running in 32-bit mode when it should be running
in 64 bit mode. To check whether gradle is running in 32-bit mode or 64-bit
mode, in the build.gradle file, dump out a few system properties as follows:<o:p></o:p></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; margin-left: 36.0pt; margin-right: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoListParagraphCxSpMiddle" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin: 0cm; mso-add-space: auto; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm; tab-stops: 45.8pt 91.6pt 137.4pt 183.2pt 229.0pt 274.8pt 320.6pt 366.4pt 412.2pt 458.0pt 503.8pt 549.6pt 595.4pt 641.2pt 687.0pt 732.8pt;">
<span style="font-family: "Courier New"; font-size: 9.0pt; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU;">println
System.properties['os.arch']<br />
println System.properties['sun.arch.data.model']<o:p></o:p></span></div>
</div>
<div class="MsoListParagraphCxSpMiddle">
<br /></div>
<div class="MsoListParagraphCxSpMiddle">
If sun.arch.data.model has a value of 32,
then it’s running in 32-bit mode.<o:p></o:p></div>
<div class="MsoListParagraphCxSpMiddle">
<br /></div>
<div class="MsoListParagraphCxSpMiddle">
Double check that the JAVA_HOME environment
variable is set to a path similar to <o:p></o:p></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; margin-left: 36.0pt; margin-right: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoListParagraphCxSpLast" style="background: #DEEAF6; border: none; margin-left: 0cm; mso-add-space: auto; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
C:\Program Files\Java\jdk1.8.0_171<o:p></o:p></div>
</div>
<div class="MsoNormal" style="margin-left: 36.0pt;">
If the path points to C:\Program
Files <b style="mso-bidi-font-weight: normal;">(x86), </b>then it is likely to be
running in 32 bit mode. In this case reinstall jdk for java in 64-bit.<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 36.0pt;">
Another symptom of running in
32-bit mode is if you try increasing the memory allocation higher than 1 GB you
may get the following error:<span style="color: black; font-size: 12.0pt; line-height: 107%; mso-ascii-font-family: Calibri; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU; mso-hansi-font-family: Calibri;"> </span><o:p></o:p></div>
<div class="MsoNormal" style="background: white; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm;">
<br /></div>
<div style="background: #DEEAF6; border: solid windowtext 1.0pt; margin-left: 36.0pt; margin-right: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-element: para-border-div; padding: 1.0pt 4.0pt 1.0pt 4.0pt;">
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="color: black; font-size: 12.0pt; mso-ascii-font-family: Calibri; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU; mso-hansi-font-family: Calibri;">C:\Users\Philip\git\lims>gradle
clean<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="color: black; font-size: 12.0pt; mso-ascii-font-family: Calibri; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU; mso-hansi-font-family: Calibri;">Error
occurred during initialization of VM<o:p></o:p></span></div>
<div class="MsoNormal" style="background: #DEEAF6; border: none; line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; mso-background-themecolor: accent1; mso-background-themetint: 51; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 1.0pt 4.0pt 1.0pt 4.0pt; padding: 0cm;">
<span style="color: black; font-size: 12.0pt; mso-ascii-font-family: Calibri; mso-bidi-font-family: "Times New Roman"; mso-fareast-font-family: "Times New Roman"; mso-fareast-language: EN-AU; mso-hansi-font-family: Calibri;">Could
not reserve enough space for 2097152KB object heap<o:p></o:p></span></div>
</div>
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-23164387549571124992017-05-14T23:20:00.000-07:002017-05-14T23:20:13.669-07:00ElasticSearch: Issues Adding a new node<h3>
The Issue</h3>
The new node was visible on the cluster but existing shards were not relocating to the new node<br />
<br />
<h3>
The steps</h3>
I had a pre-existing elasticsearch cluster of 3 nodes, and I went about adding a new node. In a round-robin fashion, I updated the elasticsearch.yml configuration of the pre-existing nodes to include the new node by updating the list of hosts and the minimum number of master nodes:<br />
<br />
<b>elasticsearch.yml</b><br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> discovery.zen.ping.unicast.hosts: ["10.0.0.1", "10.0.0.2", "10.0.0.3", "10.0.0.4"]
discovery.zen.minimum_master_nodes: 3
</code></pre>
<br />
Restarting each node, and checking the health status as follows:<br />
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@mongo-elastic-node-1 centos]# curl -XGET 10.0.0.1:9200/_cluster/health?pretty
{
"cluster_name" : "cpi",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 4,
"number_of_data_nodes" : 4,
"active_primary_shards" : 40,
"active_shards" : 71,
</code><code style="word-wrap: normal;"><span style="color: red;"><b>"relocating_shards" : 2, </b></span></code><code style="color: black; word-wrap: normal;">
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
</code></pre>
<br />
The important item to notice from above, is the bit about "relocating_shards". Here it's saying that the cluster is relocating 2 shards. To find out which shards are going where, you can check with this command:
<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="word-wrap: normal;"> [root@mongo-elastic-node-1 centos]# <b><span style="color: red;">curl -XGET http://10.0.0.9:9200/_cat/shards | grep RELO</span></b>
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7881 100 7881 0 0 318k 0 --:--:-- --:--:-- --:--:-- 334k
cpi12 2 p RELOCATING 6953804 5.8gb 10.0.0.2 cpi2 -> 10.0.0.4 fBmdkD2gT6-jTJ6k_bEF0w cpi4
cpi12 0 r RELOCATING 6958611 5.5gb 10.0.0.3 cpi3 -> 10.0.0.4 fBmdkD2gT6-jTJ6k_bEF0w cpi4
</code></pre>
<br />
Here's it's saying that cluster is trying to send shards belonging to the index called cpi12 from node cpi3 and node cpi2 to node cpi4. More specifically, it's trying to send shard #2 and shard #0 by RELOCATING them to cpi4. To monitor it's progress, I would login into cpi4 and see if the diskspace usage was going up. And here is where I noticed my first problem:<br />
<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@elastic-node-4 elasticsearch]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vdb 69G 52M 66G 1% /mnt
</code></pre>
<br />
The mounted folder where I expected to find my elasticsearch data remained unchanged at 52 MB.<br />
<br />
<h3>
Debugging</h3>
<div>
I remained stumped on this one for a long time and did the following checks:</div>
<div>
<br /></div>
<div>
<ul>
<li>The elasticsearch.yml config file for every node ensuring that discovery.zen.ping.unicast.hosts was correctly.</li>
<li>Every node could ping the new node and vice versa.</li>
<li>Every node could access ports 9200 and 9300 on the new node and vice-versa using the telnet command.</li>
<li>Every node had sufficient diskspace for the shard relocation</li>
<li>New node had the right permissions to write to it's elasticsearch folder</li>
<li style="border: 0px; font-size: 15px; margin: 0px 0px 0.5em; padding: 0px; word-wrap: break-word;">Check cluster settings: <code style="background-color: #eff0f1; border: 0px; font-family: Consolas, Menlo, Monaco, "Lucida Console", "Liberation Mono", "DejaVu Sans Mono", "Bitstream Vera Sans Mono", "Courier New", monospace, sans-serif; font-size: 13px; margin: 0px; padding: 1px 5px; white-space: pre-wrap;">curl 'http://localhost:9200/_cluster/settings?pretty'</code> and look for <code style="background-color: #eff0f1; border: 0px; font-family: Consolas, Menlo, Monaco, "Lucida Console", "Liberation Mono", "DejaVu Sans Mono", "Bitstream Vera Sans Mono", "Courier New", monospace, sans-serif; font-size: 13px; margin: 0px; padding: 1px 5px; white-space: pre-wrap;">cluster.routing</code> settings</li>
<li style="border: 0px; font-size: 15px; margin: 0px 0px 0.5em; padding: 0px; word-wrap: break-word;">Restarted elasticsearch on each node 3 times over</li>
</ul>
<div>
<span style="font-size: 15px;">However, none of the above solved the issue. Even worse, the repeated restarts of each node, managed to get my cluster into an even worse state where now some of shards became UNASSIGNED:</span></div>
</div>
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="word-wrap: normal;"> [root@mongo-elastic-node-1 bin]# curl -XGET http://10.0.0.1:9200/_cat/shards | grep UNASS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5250 100 5250 0 0 143k 0 --:--:-- --:--:-- --:--:-- 146k
.marvel-es-2017.05.13 0 p <b><span style="color: red;">UNASSIGNED</span></b>
.marvel-es-2017.05.13 0 r UNASSIGNED
.marvel-es-2017.05.14 0 p UNASSIGNED
.marvel-es-2017.05.14 0 r UNASSIGNED
cpi14 1 p UNASSIGNED
cpi13 1 p UNASSIGNED
cpi13 4 p UNASSIGNED
</code></pre>
<br />
After much browsing on the web, there was one forum that mentioned the state of the plugins on all nodes must be exactly the same as referenced from here: <a href="http://stackoverflow.com/questions/28473687/elasticsearch-cluster-no-known-master-node-scheduling-a-retry" target="_blank">http://stackoverflow.com/questions/28473687/elasticsearch-cluster-no-known-master-node-scheduling-a-retry</a><br />
<br />
<h3>
The solution</h3>
The question about the plugins got my memory jogging where I had previously installed the marvel plugin. To see what plugins are installed for each node, run the plugin command from the command-line:<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@elastic-node-3 elasticsearch]# cd /usr/share/elasticsearch/bin
[root@elastic-node-3 bin]# ./plugin list
Installed plugins in /usr/share/elasticsearch/plugins:
- license
- marvel-agent
</code></pre>
<br />
It turned out my pre-existing 3 nodes each had the <b>license</b> and <b>marvel-agent</b> plugins installed. Whereas the fresh install of the 4th node had no plugins at all. Because of this, the nodes were able to acknowledge each other, but refused to talk. To fix this, I manually removed the plugins for each node:<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@elastic-node-3 bin]# ./plugin remove license
-> Removing license...
Removed license
[root@elastic-node-3 bin]# ./plugin remove marvel-agent
-> Removing marvel-agent...
Removed marvel-agent
</code></pre>
<br />
Before I could see if shard relocation would work, I first had to assign the UNASSIGNED shards:<br />
<br />
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@mongo-elastic-node-1 elasticsearch]# curl -XPOST -d '{ "commands" : [{ "allocate" : { "index": "cpi14", "shard":1, "node":"cpi4", "allow_primary":true } }]}' localhost:9200/_cluster/reroute?pretty
</code></pre>
<br />
I had repeat this command for every UNASSIGNED shard. Checking the cluster health, I could see that there were no more unassigned shards, and that there were 2 shards currently relocating:<br />
<br />
<pre style="background: rgb(240, 240, 240); border: 1px dashed rgb(204, 204, 204); font-family: arial; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> [root@elastic-node-4 elasticsearch]# curl -XGET localhost:9200/_cluster/health?pretty
{
"cluster_name" : "cpi",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 4,
"number_of_data_nodes" : 4,
"active_primary_shards" : 40,
"active_shards" : 71,
</code><code style="word-wrap: normal;"><span style="color: red;"><b>"relocating_shards" : 2, </b></span></code><code style="color: black; word-wrap: normal;">
"initializing_shards" : 0,
"</code><code style="word-wrap: normal;"><span style="color: red;"><b>unassigned_shards" : 0,</b></span></code><code style="color: black; word-wrap: normal;">
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
}
</code></pre>
<br />
Again, checking the diskspace usage on the new node this time showed that shards were indeed relocating this time! Yay!<br />
<br />
<h3>
References</h3>
<a href="http://stackoverflow.com/questions/23656458/elasticsearch-what-to-do-with-unassigned-shards" target="_blank">http://stackoverflow.com/questions/23656458/elasticsearch-what-to-do-with-unassigned-shards</a><br />
<br />
<a href="http://stackoverflow.com/questions/28473687/elasticsearch-cluster-no-known-master-node-scheduling-a-retry" target="_blank">http://stackoverflow.com/questions/28473687/elasticsearch-cluster-no-known-master-node-scheduling-a-retry</a><br />
<br />
<a href="https://www.elastic.co/guide/en/elasticsearch/plugins/2.2/listing-removing.html" target="_blank">https://www.elastic.co/guide/en/elasticsearch/plugins/2.2/listing-removing.html</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8027951124477290753.post-67941069848815714892017-05-03T14:41:00.004-07:002017-05-04T14:43:27.799-07:00MongoDB switching to WireTiger storage engineWe were already running in production with a mongodb cluster of 3 nodes (replicated) which were running out of diskspace, each node having access to a 750 GB drive at 77% usage. The obvious solution was to expand the diskspace, but at the same time I wanted to be more efficient with the disk space usage itself.<br />
<br />
Previously we were using the storage engine callled <span style="background-color: white; color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;">MMAPv1 which had no support for compression and I wanted to switch over to the WireTiger storage engine which does have support for compression options.</span><br />
<span style="background-color: white;"><span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;"><br /></span>
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;">Here I describe the strategy I used :</span></span><br />
<span style="background-color: white;"><span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;"><br /></span>
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;">Since my mongoDB cluster was replicated, I was able to take down one node at a time to perform the switch over to WiredTiger. Once I was finished with one node, I could bring it back up, and take down the next node, and so on until all nodes were upgraded. By doing it this way, there was no downtime whatsoever from the perspective of the user.</span></span><br />
<span style="background-color: white;"><span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;"><br /></span>
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;">For each node I did the following:</span></span><br />
<span style="background-color: white; color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 16px;"><br /></span>
<br />
<ul>
<li><span style="color: white; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><b>Shutdown the mongod service</b></span></li>
<li><span style="color: white; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><b>Moved the mongo data folder</b>, which in my case was /var/lib/mongo, to another volume attached storage for backup purposes in case procedure fails.</span></li>
<li><span style="color: white; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><b>Recreate the mongo data folder</b>, in my case /var/lib/mongo and assign the appropriate permissions: <b>chown mongod:mongod /var/lib/mongo</b></span></li>
<li><span style="color: white;"><span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white;"><b>Modify the /etc/mongod.conf</b> configuration file to include the following:<span style="color: #494747;"> </span></span></span><b style="background-color: white; font-family: akzidenz, "helvetica neue", helvetica, arial, sans-serif;">storageEngine=wiredTiger</b></span></li>
<li><b style="font-family: akzidenz, "helvetica neue", helvetica, arial, sans-serif;"><span style="color: white;">Restart mongod service</span></b></li>
<li><b style="font-family: akzidenz, "helvetica neue", helvetica, arial, sans-serif;"><span style="color: white;">Check wiredTiger is configured correctly using the mongo command-line:</span></b></li>
</ul>
<pre style="background: #f0f0f0; border: 1px dashed #cccccc; color: black; font-family: "arial"; font-size: 12px; height: auto; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;"> db.serverStatus().storageEngine
{ "name" : "wiredTiger", "supportsCommittedReads" : true } </code></pre>
<br />
Now that the node is back up and running, replication will happen in the background. If you head over to your primary mongo node, and type <b>rs.status()</b> and you should see a status of <b>STARTUP2.</b><br />
Once the node has replicated successfully, repeat the same procedure for the next node.<br />
<h3>
Reference:</h3>
<br />
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white;">https://docs.mongodb.com/v3.0/release-notes/3.0-upgrade/?_ga=1.86531032.1131483509.1428671022#change-replica-set-storage-engine-to-wiredtiger</span></span><br />
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white;"><br /></span></span>
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white;">https://askubuntu.com/questions/643252/how-to-migrate-mongodb-2-6-to-3-0-with-wiredtiger</span></span><br />
<span style="color: #494747; font-family: "akzidenz" , "helvetica neue" , "helvetica" , "arial" , sans-serif;"><span style="background-color: white;"><br /></span></span>Unknownnoreply@blogger.com0