Tuesday, November 2, 2021

Docker + Logs + Fail2Ban: Originating IP addresses

 Fail2Ban is a great tool for securing our servers and relies heavily on the log files of various services. Recently, we've deployed apps using Docker containers which has been an interesting journey, but there was little in the way of documentation on how to get fail2ban working on logs produced from docker containers. So let's start with the problem:

The problem

I managed to get Docker to dump mongoDB logs to the host file system under /var/log using the driver syslog following this reference: https://techroads.org/docker-logging-to-the-local-os-that-works-with-compose-and-rsyslog/

However, any failed login attemps were logged with the local docker IP as follows:

 Nov 3 14:43:07 shiny2 docker-mongodb[18367]: {"t":{"$date":"2021-11-03T03:43:07.203+00:00"},"s":"I", "c":"ACCESS",  "id":20249,  "ctx":"conn10","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","principalName":"root","authenticationDatabase":"admin","client":"172.18.0.2:61065","result":"AuthenticationFailed: SCRAM authentication failed, storedKey mismatch"}}  
You can see that the IP address of 172.18.0.2 is a docker generated IP address and there's no point of having fail2ban blocking 172.18.0.2 from further access to the VM when the originating IP address is something completely different.

So how do we get docker to log the originating IP in the logs?

The solution

There's a little known configuration you can use in the docker-compose.yml that can tell docker to share the same network space as the host rather than trying to virtualise the network environment, shown as follows:

 version: "3.8"  
 services:  
  mongodb:  
  image : mongo  
  container_name: mongodb  
  command: [--auth]  
  environment:  
   - MONGODB_ROOT_PASSWORD=changeme  
   - MONGODB_USERNAME=someuser
   - MONGODB_PASSWORD= changeme  
   - MONGODB_DATABASE=someDB    
  volumes:  
   - /mnt/srv/databases/mongodb:/data/db  
  ports:  
   - 27017:27017  
  restart: unless-stopped  
  # Share the same network space as host to preserve IPs in log files  
  network_mode: host  
  logging:  
   driver: syslog  
   options:  
    tag: docker-mongodb  
 networks:  
  default:  
  external: true  
  name: db-net  
I hope this helps someone save hours of searching like I did!

Sunday, September 12, 2021

Apple iOS upload to SAMBA drive fails with "The operation couldn't be completed. Operation cancelled"

 Apple iOS 14.5 and upwards (so far) has a introduced a bug preventing apple users from uploading or creating files on their local samba drives on the network. The upload will start but eventually fail with an error popup:

The operation couldn't be completed. Operation cancelled

If you have access to the samb configuration file (smb.conf) then you can apply a known workaround. In my instance, my samba is hosted on an Ubuntu machine. Edit the file found at /etc/samba/smb.conf and under the [global] section add the following 2 lines:


 [global]
 vfs objects = fruit streams_xattr  
 fruit:nfs_aces = no  

Then restart samba

service smbd restart
And you're good to go!

Monday, June 21, 2021

Springboot + Vue.js + Azure Active Directory Authentication

Springboot + VueJS + Azure AD Authentication

 

This is a configuration guide on how to setup a SpringBoot application running as a backend API with a VueJS frontend, authenticated using Microsoft’s Azure Active Directory (AD).

 In this case, we are using Azure only for authentication, whereby staff and students of the university, can use their own credentials to access our custom-built platforms.

The process by which authentication occurs step-by-step can be summarised in the diagram below:



In the above diagram we can link the following:

Client: VueJS client application

Resource Server: Springboot API

Authorization server: Azure Active Directory

 

This approach relies on Azure configuration in both the front-end VueJS app and Springboot API app

1)      When a user first visits your secured VueJS application, the user needs to be identified and is redirected to the Azure authentication server for login. Once the user has been identified, the access token can be requrested from the Authorization server (in this case Azure).

2)      The access token is stored internally in the VueJS app and used later in subsequent calls to the API as part of the ‘Bearer’ header. The API (Resource server), accepts the token and forwards it to Azure for validation of the access Token.

3)      Azure responds to the API confirming the access token is valid, including what roles the user has.

4)      The API checks the user roles have access to the specific resource (API) being requested.

5)      Once access has been confirmed, the API returns the result to the VueJS client.

In this configuration, the API is no longer responsible for generating tokens. All tokens are managed by Azure. The API simply passes tokens around for validation.

Azure

https://portal.azure.com
https://portal.azure.com/#blade/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade

Reference: https://devblogs.microsoft.com/azure-sdk/vue-js-user-authentication/

Goto “Active Directory”  


Before we can configure our VueJS and Springboot app, we need to create a new App in Azure. There are already many online instructions on how to do this, so I will cut to the chase:

Registration



App IDs

Once the new App has been registered we need to take note of the Client ID and Tenant ID:



 

 

Redirect URIs

Add the redirect URIs hosting your VueJS app.



The URIs should be for your VueJS app. In my case, localhost:3001 was for VueJS.

API Permissions

Make sure the App has the following API permissions set:

 

App Roles

For each app we can create our own roles, and assign users to those roles.



Here, we’ve created 2 roles with values ROLE_ADMIN and ROLE_RESEARCHER. The allowed member types should be set to ‘Users/Groups’.

Manifest (Might be optional)

In the manifest file set the accessTokenAcceptedVersion to 2

Enterprise Application Mode

Switch to Enterprise Application mode by searching for your App

Add users and assign roles



 



Pick a user and pick a role, and save.

 

And that’s it for the Azure setup.


 

VueJS + MSAL 2.0

Reference: https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/lib/msal-browser

Reference: https://docs.microsoft.com/en-us/azure/active-directory/develop/scenario-spa-app-registration

Reference: https://docs.microsoft.com/en-us/azure/active-directory/develop/id-tokens

Microsoft has released a javascript library for developers to authenticate users with Azure Active Directory called MSAL.js

In your existing VueJS application, from the command line, import the following libraries.

npm I @azure/core-http

npm I @azure/msal-browser

 

At the time of writing this document, the versions were:

    "@azure/core-http""^1.2.6",

    "@azure/msal-browser""^2.14.2",

 

Under services folder, create a file called auth-azure.service.js from:

https://github.com/Philip-Wu/VueJS-MSAL/blob/main/auth-azure.service.js

Modify the file to set the clientId, tenantId and authority url in the config. The authority URL may be different depending on how your organization is setup. For more information on authority: https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration#authority

In my case the authority was configured as:

authority: 'https://login.microsoftonline.com/<tenantId>,

 

Where the URL is suffixed with the tenantId.

Please take note of the defaultScope used for acquiring tokens.

This is super important. Otherwise, we get an error about 'Invalid signature' due to receiving an access token in v1.0 format.

The backend API will attempt to validate the token using a 2.0 endpoint, which is not suited for an v1.0 access token. So to force v2.0 accessToken, we use the ./default. This was not mentioned in any formal documentation

 

The defaultScope should be <your_client_id>/.default in order to get the v2.0 format for access tokens. This should already be coded in the function from the github file. This was not well documented by Azure.

The .env file should contain the redirect URI that should be the same as what was configured in Azure earlier:

VUE_APP_AZURE_AUTH_REDIRECT_URI='http://localhost:3001/'

It must be prefixed with VUE_APP_ in order for VueJS to make it available to the application. Otherwise, it will be undefined.


In the router/index.js file, when the user clicks on the login button, we can invoke the auth-azure script:

        {
            path: "/login",
            name: "login",
            component: Login,
            beforeEnter: async (tofromnext=> {                
                console.log('Login route, user: '+authAzure.user());
                if(! authAzure.user()) {
                    if (app != undefined) {
                        authAzure.appSignIn();
                    } 
                } else {
                    console.log('already authenticated user:');
                }

                next();
            }                     }, 

Further down in the router/index.js file, we want to stored a redirectPath in the session if a secured page was requested:

router.beforeEach((tofromnext=> {
  const publicPages = ['/login''/register''/','/home'];
  const authRequired = !publicPages.includes(to.path);
  const loggedIn = authAzure.isLoggedIn();
  // trying to access a restricted page + not logged in
  // redirect to login page
  if (authRequired && !loggedIn) {
    sessionStorage.setItem('redirectPath'to.path);
    
    next('/login');
  } else {
    next();
  }
});

 

In main.js, we handle the created() event by checking if the user is already logged in:

var app = new Vue({
    router,
    store,
    created() {
      console.log('app created')
      authAzure.init()
    },    
    async mounted() {
      console.log('app mounted')

      let redirectPath = sessionStorage.getItem('redirectPath');      
      if (authAzure.isLoggedIn() && redirectPath) {

        // Define function to be used as callback
        let redirectFunc = function() {
          console.log('redirecting to 'redirectPath)
          sessionStorage.removeItem('redirectPath')
          router.push(redirectPath)  
        }

        if (! authAzure.waitingOnAccessToken) {
          redirectFunc()
        } else {
          // Register callback to be executed when accessToken has been assigned
          authAzure.accessTokenCallbacks.push(redirectFunc)
        }
      }

    },
    render: h => h(App),
    renderError(herr) {
      return h('pre', { style: { color: 'red' }}, err.stack)
    } 
  });

 

 

Create a file called auth-header.js that will be used as a function for supplying the Bearer Authorization header with the acquired accessToken:

function authHeaderAzure() {
    let accessToken = authAzure.accessToken;
    console.log(app+'using token: '+accessToken);

    if (accessToken) {
        return { Authorization: 'Bearer ' + accessToken };
    } else {
        return {};
    }

}

 

Create a file auth.module.js to handle the dispatches from auth-azure.service.js. If the user was attempting to access a secured page directly, then they are redirected the requested page immediately after authentication.

import authAzure from '../services/auth-azure.service';
import app from '../main'

/**
 * Used for rendering the navigation componenet, Nav.vue, to manage the state of being Logged in vs Logged out.
 * Using the Vuex.store we can trigger Nav.vue to re-render upon logging in or out.
 */

const user = authAzure?.user()

const initialState = user
    ? { loggedIn: true  }
    : { loggedIn: false };

export const auth = {
    namespaced: true,
    state: initialState,

    mutations: {
        loginSuccess(state) {
            console.log('mutation loginSuccess')
            state.loggedIn = true;            
        },
        loginFailure(state) {
            state.loggedIn = false;
        },
        logout(state) {
            console.log('mutation logoutSuccess')
            state.loggedIn = false;
        },
        
    },

    actions: {
        loginSuccess({commit}) {
            commit('loginSuccess')

            // Redirect user if a page was loaded directly in the browser
            let redirectPath = sessionStorage.getItem('redirectPath');
            console.log('redirectPath: '+redirectPath)
            if (redirectPath) {
                sessionStorage.removeItem('redirectPath');
                app.$router.push(redirectPath)
            }

        },
        logout({commit}) {
            commit('logout')
        },
        loginFailure({commit}) {
            commit('loginFailure')
        }
    }
};

 

 

Troubleshooting

Can use the following tool to decode an access token. Take carefully notice of the token version number:

https://jwt.ms/


 

Springboot + Azure authentication

Reference: https://docs.microsoft.com/en-us/java/api/overview/azure/spring-boot-starter-active-directory-readme?view=azure-java-stable (accessing a Resource server section)

Reference: https://github.com/Azure/azure-sdk-for-java/tree/azure-spring-boot-starter-active-directory_3.5.0/sdk/spring/azure-spring-boot-samples/azure-spring-boot-sample-active-directory-resource-server

Reference: https://developer.okta.com/blog/2019/06/20/spring-preauthorize

 

To setup our Springboot app as a Resource server, we add the following Azure libraries to our dependency to our pom.xml file:

<dependency>
   <groupId>
com.azure.spring</groupId>
   <artifactId>
azure-spring-boot-starter-active-directory</artifactId>
   <version>
3.5.0</version>
</dependency>
<dependency>
   <groupId>
org.springframework.boot</groupId>
   <artifactId>
spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>

 

Then we need to configure our springboot security by creating the following file:

@Slf4j
@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity
(prePostEnabled = true)
class WebSecurityConfig extends AADResourceServerWebSecurityConfigurerAdapter{

   
@Autowired
   
UserDetailsServiceImpl userDetailsService

   
@Autowired
   
private AuthEntryPointJwt unauthorizedHandler

   
@Override
   
protected void configure(HttpSecurity http) throws Exception{
       
super.configure(http)

               
.antMatchers("/","/v1/var/**","/home","/signin","/login","/hash", "/signup", "/explorer/**").permitAll()
               
.anyRequest().authenticated()     } } }} 

Next we need to configure our Azure app by providing the tenant-id and client-id in the application.yml file:

azure:
 
activedirectory:
   
tenant-id: <azure app tentant id>
   
client-id: <azure app client id>

 

If you want to debug an azure related error, I would also highly suggest setting the root logging level to DEBUG in the application.yml. This helped me to resolve a misleading error message:

logging:
 
file:
   
path: logs
 
level:
   
root: DEBUG

 

Before we can define the role access for our API in the controller, we need to work out how the roles are renamed by Azure (or the library). This is where I spent most of my time as none of this was mentioned in any of the official documentation or any forums. This is where the logging level of DEBUG was handy.

If the roles do not match exactly between Azure and the Springboot API, then on the browser we may see network errors when debugging using the “Developer tools” of the browser:



Here we can see the error says “insufficient_scope” and “The request requires higher privileges than provided by the access token”. For me this was misleading, suggesting that the issue was related to the Azure API Permissions, but that was not the case. The API permission ‘User.Read’, should be sufficient privileges for authentication and accessing our API. Rather than an API permission issue, it was really a ROLE configuration issue.

On the API server, we can see the following logs:

2021-06-21 11:42:38,284 {HH:mm:ss.SSS} [http-nio-8081-exec-1] DEBUG o.s.s.o.s.r.w.BearerTokenAuthenticationFilter - Set SecurityContextHolder to BearerTokenAuthentication [Principal=com.azure.spring.aad.webapi.AADOAuth2AuthenticatedPrincipal@4581efe6, Credentials=[PROTECTED], Authenticated=true, Details=WebAuthenticationDetails [RemoteIpAddress=0:0:0:0:0:0:0:1, SessionId=null], Granted Authorities=[SCOPE_User.Read, APPROLE_ROLE_ADMIN]]

We can see that Azure is prefixing my configured roles with APPROLE_

This means, in my controllers, I need to specify my role access using APPROLE_ROLE_ADMIN as follows:

@Slf4j
@RestController
@RequestMapping
(path="/v1/var")
@PreAuthorize("hasAnyAuthority('APPROLE_ROLE_ADMIN', 'APPROLE_ROLE_RESEARCHER')")
class VariantAnnotationController {

Here we use the @PreAuthorize annotation and the hasAnyAuthority method to control access to the API based on the Azure roles previously configured.


 

Other notes:

Initially I had tried to use the VueJS plugin as a wrapper for the MSAL.js client library, but I had trouble getting that to work, perhaps because it was initially designed for an older version of the MSAL library and hasn’t been updated for a while. Just for reference the vueJs plugin is the ‘vue-msal’ plugin: https://github.com/mvertopoulos/vue-msal

 

  

Thursday, May 20, 2021

"cannot read property prototype of undefined" lb4 repository decorator

 Using Loopback4, I was trying to create new repositories and noticed afterwards when trying to start the app, I had the following error:

 TypeError: Cannot read property 'prototype' of undefined  
   at Object.repository (C:\sts-workspace\rodentity-nodejs\node_modules\@loopback\repository\src\decorators\repository.decorator.ts:145:53)  
   at Object.<anonymous> (C:\sts-workspace\rodentity-nodejs\src\services\unfinished-genotyping-plates.service.ts:17:6)  
   at Module._compile (internal/modules/cjs/loader.js:778:30)  
   at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)  
   at Module.load (internal/modules/cjs/loader.js:653:32)  
   at tryModuleLoad (internal/modules/cjs/loader.js:593:12)  
   at Function.Module._load (internal/modules/cjs/loader.js:585:3)  
   at Module.require (internal/modules/cjs/loader.js:692:17)  
   at require (internal/modules/cjs/helpers.js:25:18)  
   at Object.<anonymous> (C:\sts-workspace\rodentity-nodejs\src\services\index.ts:3:1)  
   at Module._compile (internal/modules/cjs/loader.js:778:30)  
   at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)  
   at Module.load (internal/modules/cjs/loader.js:653:32)  
   at tryModuleLoad (internal/modules/cjs/loader.js:593:12)  
   at Function.Module._load (internal/modules/cjs/loader.js:585:3)  
   at Module.require (internal/modules/cjs/loader.js:692:17)  

The error message is not very helpful. But after debugging it, I worked I had a missed a step when creating the repository. I needed to add a line to the application.ts file to register the newly created repositories as follows:

 this.repository(MouseRepository);  

Sunday, November 15, 2020

Loopback LB4, mysql SSL configuration

 I had a tough time trying to find documentation on how to configure loopback 4 to use SSL certs for communicating with a mySQL backend. So here's how I did it:



    dataSource: {  
     name: "mysql",  
     connector: "mysql",  
     // url: process.env.DS_URL, // don't use URL because SSL doesn't work with it  
     host: process.env.DB_HOST,  
     port: 3306,  
     ssl: {  
      rejectUnauthorized: false,  
      ca: fs.readFileSync(process.env.mysqlSSLCAcert, { encoding: 'utf8', flag: 'r' }),  
      key: fs.readFileSync(process.env.mysqlSSLClientKey, { encoding: 'utf8', flag: 'r' }), //client  
      cert: fs.readFileSync(process.env.mysqlSSLClientCert, { encoding: 'utf8', flag: 'r' }), // client  
     },  
     user: process.env.DB_USERNAME,  
     password: process.env.DB_PASSWORD,  
     database: "facility"  
    },  

Thursday, May 28, 2020

RabbitMQ proxied through apache httpd web server

Problem

RabbitMQ wasn't allowing us to add exchanges through the web console. Clicking on the button resulted in the PUT request being sent to apache, but apache wasn't forwarding the traffic onto rabbitMQ.

This was confirmed by checking the apache logs. Indeed the request was received, but in the rabbitMQ logs, there was no change in activity. This would a suggest a problem at the apache level.

At the browser level, debugging shows a 404 status code was returned, which suggests that apache wasn't able to find handler for the request.

Solution


Reference: https://serverfault.com/questions/639327/rabbitmq-behind-apache-mod-proxy-not-resolving-deep-link

Modify your httpd.conf or ssl.conf to have the following
Edit /etc/rabbitmq/rabbitmq.conf and add
 management.path_prefix = /mq  


Edit /etc/httpd/conf.d/ssl.conf and add the following rules:
   AllowEncodedSlashes NoDecode  
   ProxyPass /mq http://localhost:15672/mq nocanon  
   ProxyPassReverse /mq http://localhost:15672/mq  

It's important to note that a trailing slash is required when entering the URL path in the browser. For example:

https://myhost/mq/

Friday, February 14, 2020

How to remove bloatware from Oppo Reno Z

Its unbelievable how much bloatware apps are installed by defualt on Oppo phones. But there is a way to remove them using some simple command-line tools.

This article describes how to remove the bloatware installed by default on a new Oppo Reno Z using the adb tool.

Follow this guide to install ADB: 


To install ADB go here:


In Windows 10 using powershell, launch ADB as follows

cmd /c adb shell

 Most of the bloatware is installed under the package "com.coloros"

pm list packages | grep coloros

Here are the commands I used for removing the bloatware apps:

pm uninstall -k --user 0 com.coloros.gamespace
pm uninstall -k --user 0 com.coloros.weather.service
pm uninstall -k --user 0 com.coloros.weather2
pm uninstall -k --user 0 com.coloros.gallergy3d
pm uninstall -k --user 0 com.coloros.musiclink
pm uninstall -k --user 0 com.coloros.phonemanager
pm uninstall -k --user 0 com.coloros.calculator
pm uninstall -k --user 0 com.coloros.screenrecorder
pm uninstall -k --user 0 com.coloros.alarmclock
pm uninstall -k --user 0 com.coloros.compass2
pm uninstall -k --user 0 com.coloros.oppomultiapp
pm uninstall -k --user 0 com.coloros.soundrecorder
pm uninstall -k --user 0 com.coloros.oppopods
pm uninstall -k --user 0 com.coloros.onekeylockscreen
pm uninstall -k --user 0 com.coloros.video
pm uninstall -k --user 0 com.coloros.


That cleaned up quite a bit and removed icons from the screen.

Unfortunately, there were some apps I couldn't remove such as the Oppo Contacts, and Clone Phone.
I'd be interested to hear if anybody figured out how to remove them, or other bloatware found on their phone.

Restore uninstalled apps

The apps are not really deleted, they're just hidden. To view a list of all packages including the uninstalled apps use

pm list packages -u | grep coloros

To restore an app in adb shell

cmd package install-existing com.coloros.usbselection


Good luck