My architecture is simple. Services: web > datastore.

All I want to do is inject a 5s delay into any requests going to the datastore, so I've created a VirtualService to do this:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: datastore-virtualservice
spec:
  hosts:
  - datastore
  http:
  - fault:
      delay:
        percentage:
          value: 1
        fixedDelay: 5s
    route:
    - destination:
        host: datastore

Some other info if it helps:

kubectl get services
NAME             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
datastore        ClusterIP   10.106.54.111    <none>        27017/TCP   23h
kubernetes       ClusterIP   10.96.0.1        <none>        443/TCP     2d18h
web              ClusterIP   10.106.162.190   <none>        3000/TCP    23h

YAML Files

web-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: web
  name: web
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: web
    spec:
      containers:
      - image: adamgardnerdt/customnode
        name: web
        ports:
        - containerPort: 3000
        resources: {}
      restartPolicy: Always
status: {}

web-service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: web
  name: web
spec:
  ports:
  - name: "3000"
    port: 3000
    targetPort: 3000
  selector:
    io.kompose.service: web
status:
  loadBalancer: {}

datastore-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: datastore
  name: datastore
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: datastore
    spec:
      containers:
      - image: adamgardnerdt/custommongo
        name: datastore
        ports:
        - containerPort: 27017
        resources: {}
      restartPolicy: Always
status: {}

datastore-service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: datastore
  name: datastore
spec:
  ports:
  - name: "27017"
    port: 27017
    targetPort: 27017
  selector:
    io.kompose.service: datastore
status:
  loadBalancer: {}

Ingress Gateway and VirtualService

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: quotations-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: quotations-virtualservice
spec:
  hosts:
  - "*"
  gateways:
  - quotations-gateway
  http:
  - match:
    - uri:
        exact: /
    rewrite:
      uri: /
    route:
    - destination:
        host: web
        port:
          number: 3000

Edit

My web service is a nodeJS based web server. I've tried this with an httpd server and the delay works fine. Therefore it's something to do with the async nature of node.

My confusion grows. Regardless of the async nature of nodeJS, the page is only rendered to the user within the renderPage() function. So...

1) db.collection('quotes').find().toArray((error, quotes)) is executed.

2) Istio sees the call going from web to datastore and delays it for X seconds.

3) The call succeeds and finally renderPage is called.

4) The page is rendered (with DB content) after the Istio-induced delay.

What's actually happening is that the page (with DB content) is rendering immediately. I don't know how / why this is happening...

Code for my web image:

const express = require('express');
const filesystem = require('fs');
const app = express();

var MongoClient = require('mongodb').MongoClient;
var MongoURL = "mongodb://datastore:27017/";

var defaultQuotes = [{ "author": "no quotes", "content": "No content. Is MongoDB down?"}];

function renderPage(error, quotes, res) {
  res.render(__dirname + '/index.ejs', { quotes: quotes });
}

app.set('view engine', 'ejs');

app.listen(3000, function() {
  console.log('listening on 3000')
});

app.get('/', (req, res) => {
  // Attempt a connection to MongoDB
  MongoClient.connect(MongoURL, function(err, client) {
    console.log('connected to MongoDB');
    try {
      var db = client.db('quotesDB');
      // Get all quotes and then call the renderPage function.
    db.collection('quotes').find().toArray((error, quotes) => renderPage(error, quotes, res));

    // Close the connection to Mongo
    client.close();
    }
    catch (e) {
      console.log('exception caught');
      renderPage(null, defaultQuotes, res);
    }
  });

});