support TypeScript for backend functions

Is your feature request related to a problem? Please describe.
I use the TypeScript generation features for my front-end client, but am frustrated that any of the Lambda functions I create via amplify add api or amplify add function must be written in vanilla JavaScript. This also makes it very difficult to share code between the backend and frontend (note that many of the auto-generated files created by amplify cli use syntax like export default blah that is invalid for use in the backend files).
Describe the solution you'd like
Switch to using tsc to compile the files found in amplify/backend/function/*/src/ so that we can opt-in to type-safety by changing our file extension from .js to .ts or .tsx, similar to how Create React App has handled this in version 2.0.0+.
Describe alternatives you've considered
Alternatively, add (or document) how we can hook into the build process for amplify/backend/function/* so we can inject a TypeScript transpilation step ourselves.

Have you considered using the "postinstall" hook in npm? I have not tried using it but am about to give it a shot.
Amplify uses a "amplify function build" command and the docs say it calls npm install under the hood. The npm docs say that npm run postinstall is called automatically after npm install, so it looks promising. This might require installing typescript + build tools on the build machine without utilizing the function's package.json, but that's not too bad.

This looks like a promising approach but I have a stumbling block that I haven't been able to figure out.
This might look like a long procedure, but it should take about 2 minutes and once it is configured, it should be easy to maintain.
WARNING: THIS WILL OVERWRITE CODE IN YOUR FUNCTION'S "src" directory (eg. amplify/backend/function/myFunction/src)
Strategy:

Create a sister directory to amplify/backend/function/myFunction/src called function/myFunction/tsc-src
Create a file at function/myFunction/src/postinstall.sh:

#!/bin/bash

cd ../tsc-src
npm run build

Make postinstall.sh executable (on my mac the command is): chmod u+x postinstall.sh

Update function/myFunction/src/package.json with the postinstall script:

{
  "name": "myFunction",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "UNLICENSED",
  "scripts": {
    "postinstall": "./postinstall.sh"
  }
}

In function/myFunction/tsc-src (eg. cd function/myFunction/tsc-src) run the command tsc --init

Edit function/myFunction/tsc-src/tsconfig.json to include:

    "outDir": "../src",                        /* Redirect output structure to the directory. */

Write your typescript in the tsc-src folder. If you have javascript in your src folder, you should copy it to a different folder because this setup might overwrite that src folder when you build the lambda function. I would recommend starting with a very simple typescript file so you know the build process is configured properly

Install build dependencies You will need to do this on every build machine!

yarn global add typescript

WARNING: THIS NEXT STEP WILL OVERWRITE CODE IN YOUR FUNCTION'S "src" directory (eg. amplify/backend/function/myFunction/src)

To test everything:

// amplify function build myFunction
amplify function invoke myFunction
... You should see javascript files being generated in amplify/backend/function/myFunction/src ...

With the above strategy, I was able to import and configure Amplify. I have not tested it in the backend, yet. You might run into a typescript error issue with aws-exports.js, I solved that by adding this line to postinstall.sh. The relative path should actually be the same for your project.:
cp ../../../../../src/aws-exports.js ./aws-exports.ts

I noticed that you have to update package.json everytime before calling amplify function build for the typescript to compile into JS. I know we are working around this problem and it would be nice if Amplify CLI automatically compiles Typescript to JavaScript out of the box.

Or if at least they provide us with a hook to run a npm script or similar... :)

FWIW I am building an amplify app with Clojurescript and would love the option to easily write my node.js functions in the same language as the rest of my stack. If support for typescript is considered I hope with that would come in the form of a way to support all compile to js languages.

Is it possible to test this locally?
Referring to the documentation you have to user amplify push.

@kaustavghosh06 I had the exact same 3 problems as @mrcoles when reading the documentation.

You can read about adding lambda functions in Typescript as a part of this blog - https://servicefull.cloud/blog/amplify-ts/

+1

Has anyone tried TypeScript recently in Amplify Lambda functions with Promises?  I'm receiving errors that I can't get past.
If I try to use Promise directly (e.g. new Promise()), I get the following error:
index.ts:6:38 - error TS2585: 'Promise' only refers to a type, but is being used as a value here. Do you need to change your target library? Try changing the `lib` compiler option to es2015 or later.

6     const wait = (ms: number) => new Promise(res => setTimeout(res, ms));
                                       ~~~~~~~

If I try to use async/await:
error TS2468: Cannot find global value 'Promise'.
index.ts:91:35 - error TS2705: An async function or method in ES5/ES3 requires the 'Promise' constructor.  Make sure you have a declaration for the 'Promise' constructor or include 'ES2015' in your `--lib` option.

91     const updateDocument = async () => {

Even in the blog https://servicefull.cloud/blog/amplify-ts/, if I use that example (with their tsconfig.json) with the following code:
import { APIGatewayProxyHandler } from 'aws-lambda';
import 'source-map-support/register';

export const handler: APIGatewayProxyHandler = async (event, _context) => {
    console.log(event);
    const wait = (ms: number) => new Promise(res => setTimeout(res, ms));
    console.log('data', wait);
    return {
        statusCode: 200,
        body: JSON.stringify({
            message: 'Amplify function in Typescript!',
            input: event
        })
    };
};

It throws the same error as the first one above.  I've tried this with numerous tsconfig.json configurations and none of them seem to make a difference.

Nevermind on the post above.  Turns out that in the blog post https://servicefull.cloud/blog/amplify-ts/, the tsc ./*.ts command in package.json skips the tsconfig.json that you created and uses the default settings, so I switched it to just 'tsc' and it works fine (or you can use tsc --project tsconfig.json).

@kaustavghosh06 - is it possible to run this same behaviour when running amplify mock too? I have setup some build scripts that run webpack on my files so that I can now have Typescript and use shared libraries etc. However, the scripts are only run with amplify push. In the time being I will create my own little scripts to handle this but would be good if it was baked in.

@kaustavghosh06 - is it possible to run this same behaviour when running amplify mock too? I have setup some build scripts that run webpack on my files so that I can now have Typescript and use shared libraries etc. However, the scripts are only run with amplify push. In the time being I will create my own little scripts to handle this but would be good if it was baked in.

@ChrisSargent  +1
amplify mock function does not execute Build Options for functions properly

FWIW, I used package script to achieve this for now, something along the lines of: "mock:api": "concurrently \"amplify mock api\" \"tsc --watch\"". I also actually treat each function as its own yarn workspace and put my real source code in a function/functionname/lib folder. This then compiles with webpack in to the src folder which is where amplify needs it to be. The code in the src folder is completely bundled and can be optimised / tree shaken etc. with webpack. So I have a folder structure like this:
function
- functionname
-- lib
-- index.ts // exports my handler
-- src
-- index.js // minified & bundled
-- package.json // empty
- package.json // has the dependencies for index.ts in /lib

Then, because of Yarn workspaces, I can also import local shared packages - which, of course, get bundled in to the final index.js code.

My 2 cents on running amplify with Typescript... (be prepared for some code copy and pasting)

on the entire application, make use of yarn workspaces... I configure my package.json workspaces like this

"workspaces": {
    "packages": [
      "amplify/backend/function/**/src"
    ],
    "nohoist": [
      "**"
    ]
  },

add a root tsconfig.json like this

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es2017",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "types": ["node"],
	"module": "commonjs",
	"paths": {

	},
	"baseUrl": "."
  },
  "exclude": [
	  "node_modules",
	  "**/build",
	  "**/dist"
	],
}

Everytime you add a lambda function, edit the {name}-cloudformation-template.json and set the Handler property not to reference your ts file but your final built js file. (build/index.handler instead of just index.handler)

"Handler": "build/index.handler",

inside the src folder for the lambda, edit the lambda package.json
{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the src folder for the lambda, add a tsconfig file that extends your root tsconfig:
{
  "extends": "../../../../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./build",
    "rootDir": "./"
  }
}

update your root package.json for the entire amplify project to compile your new lambda on push

"scripts": {
	"amplify:lambdaName": "cd amplify/backend/function/lambdaName/src && npx tsc"
  },

Everytime you add a lambda layer:

inside the lib/nodejs file, edit the package.json to be like so
{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the lib/nodejs file, add a tsconfig.json file
{
	"extends": "../../../../../../tsconfig.json",
	"compilerOptions": {
	  "outDir": "./build",
	  "rootDir": "./",
	  "declaration": true
	}
}

** note the additional declaration output **
edit your root package.json to build the lambda layer on amplify push by add this under the scripts section
"amplify:layerName": "cd amplify/backend/function/layerName/lib/nodejs && npx tsc"

edit your root tsconfig.json to add a reference to you declaration file from the lambda layer
"paths": {
		"/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile": ["./amplify/backend/function/layerCommonDynamoDb/lib/nodejs/build/declarationFileNameFromLambda.d.ts"]
	},

** note: exampleOutputJsFileFromInLambdaLayerBuildFile is the final build file from your lambda layer... fileName is important as it has to match aws lambdas /opt/nodejs folder structure to work on AWS correctly otherwise your directory references will be wrong **
you can now use your lambda layer in your lmbda functions:
import commonLib from '/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile'

I haven't tried using this with amplify mock, i think a lot of work needs to be done on the local amplify mocking / testing environment anyway so rather use a multi environment AWS workflow... add the sourcemap import to the top of your lambda as well for better debugging experience.

@danielblignaut thank you so much. I had this all working in a very hacky manner. But this is clean and simple, appreciate the advice.

I'm actually against adding this functionality until Typescript is supported natively by AWS Lambda- hear me out...
The ideal scenario would be that typescript would be pushed up as is.
We'd then get type support directly inside the Lambda editor on the AWS Console.
It would also really suck to have Amplify spend time on this to then see Typescript support pop up in AWS Lambda.

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

@r0zar I think it's a very low / non-existant priority for the lambda team as there is no "typescript" runtime in reality and even if so, it would probably be less performant if you want to run something like ts-node v building your package and running it directly as building with webpack gives you tree shaking, and other features to increase speed and reduce package size. You can also get runtime type support (for stack traces) with source maps already.
personally, as a typescript developer, I've moved away from Amplify and have instead adopted AWS CDK which is another infrastructure as code library written by the AWS team. This library is based in typescript (although there are other language variations) and all the latest features come to typescript first. I also use lerna for a mono-repo structure to hold all my lambda's and build their typescript packages independently. With CDK you can choose the file location of your lambda code so I just add my lambda's as dependency's to the cdk package and use require.resolve to find their built code in the node_modules and deploy. Honestly works like a dream, does not feel hacky at all and personally my experience with CDK as a typescript dev has been great especially on bigger projects.

@danielblignaut, I am curious about your implementation, do you have an open source example?

My 2 cents on running amplify with Typescript... (be prepared for some code copy and pasting)

on the entire application, make use of yarn workspaces... I configure my package.json workspaces like this

"workspaces": {
    "packages": [
      "amplify/backend/function/**/src"
    ],
    "nohoist": [
      "**"
    ]
  },

add a root tsconfig.json like this

{
  "compilerOptions": {
    "sourceMap": true,
    "target": "es2017",
    "moduleResolution": "node",
    "esModuleInterop": true,
    "types": ["node"],
	"module": "commonjs",
	"paths": {

	},
	"baseUrl": "."
  },
  "exclude": [
	  "node_modules",
	  "**/build",
	  "**/dist"
	],
}

Everytime you add a lambda function, edit the {name}-cloudformation-template.json and set the Handler property not to reference your ts file but your final built js file. (build/index.handler instead of just index.handler)

"Handler": "build/index.handler",

inside the src folder for the lambda, edit the lambda package.json
{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the src folder for the lambda, add a tsconfig file that extends your root tsconfig:
{
  "extends": "../../../../../tsconfig.json",
  "compilerOptions": {
    "outDir": "./build",
    "rootDir": "./"
  }
}

update your root package.json for the entire amplify project to compile your new lambda on push
"scripts": {
	"amplify:lambdaName": "cd amplify/backend/function/lambdaName/src && npx tsc"
  },

Everytime you add a lambda layer:

inside the lib/nodejs file, edit the package.json to be like so
{
  "name": "uniqueName",
  "version": "2.0.0",
  "description": "Lambda function generated by Amplify",
  "main": "index.js",
  "license": "Apache-2.0",
  "dependencies": {
    "source-map-support": "^0.5.16"
  },
  "devDependencies": {
    "@types/aws-lambda": "^8.10.36",
    "@types/node": "^12.12.14",
    "typescript": "^3.7.2"
  }
}

inside the lib/nodejs file, add a tsconfig.json file
{
	"extends": "../../../../../../tsconfig.json",
	"compilerOptions": {
	  "outDir": "./build",
	  "rootDir": "./",
	  "declaration": true
	}
}

** note the additional declaration output **
edit your root package.json to build the lambda layer on amplify push by add this under the scripts section
"amplify:layerName": "cd amplify/backend/function/layerName/lib/nodejs && npx tsc"

edit your root tsconfig.json to add a reference to you declaration file from the lambda layer
"paths": {
		"/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile": ["./amplify/backend/function/layerCommonDynamoDb/lib/nodejs/build/declarationFileNameFromLambda.d.ts"]
	},

** note: exampleOutputJsFileFromInLambdaLayerBuildFile is the final build file from your lambda layer... fileName is important as it has to match aws lambdas /opt/nodejs folder structure to work on AWS correctly otherwise your directory references will be wrong **
you can now use your lambda layer in your lmbda functions:
import commonLib from '/opt/nodejs/build/exampleOutputJsFileFromInLambdaLayerBuildFile'

I haven't tried using this with amplify mock, i think a lot of work needs to be done on the local amplify mocking / testing environment anyway so rather use a multi environment AWS workflow... add the sourcemap import to the top of your lambda as well for better debugging experience.

@danielblignaut thanks so much for your reply. I found it incredibly helpful. I am finally able to use my typescript lambda layer properly.
Could you please elaborate on your final comment about not using this with amplify mock. (It doesn't work with amplify mock by the way, as soon as I import my lambda layer library and run amplify mock it just hangs and then times out).
So I am just wondering how you test your code before pushing to amplify. Do you still test it locally? Is there a way to test this locally? (A better way than amplify mock???) What exactly do you mean by multi environment AWS workflow?
Thanks again :)

Hey @ziggy6792 ,
I can’t quite remember the reasoning behind the last comment. But to provide some further points:

I actually ended up avoiding layers all together. I found amplify struggled to maintain correct layer versioning with the remote AWS account leading to me deploying lambda functions referencing out of date lambda layers, etc. I opened up a GitHub issue on Amplify code base so should be able to find if it’s been resolved or not (this was when layers first launched).
I’ve moved away from using layers in JavaScript. My reasoning: I feel that layers are great to provide common library access for most languages! However, JavaScript has awesome tooling from its community and I feel that for that reason, layers is least advantageous to JavaScript language v others because of yarn, lerna, etc. I think a better JavaScript and especially typescript approach is to use lerna in a monorepo structure, create the lambda layer as a normal package in that monorepo and require it in your package.
As a typescript developer, I’ve moved away from amplify and now use CDK. I recommend for typescript devs in particular you check this library out and especially for bigger projects. You have to write a little more code for your infrastructure but you gain more flexibility in project design and you win first class typescript support.
Testing is always hard to emulate. It’s really important that you test on an AWS (non production stage). No tool I’ve used: localstack, AWS local step functions, AWS dynamodb actually recreates the real AWS service without some limitation or bug and there’s a MASSIVE gap in development experience here. Personally, I test all my graphql APIs by writing queries that test the appsync server directly. Last time I checked, and that’s a while ago, amplify actually just runs a local server that proxies the remote appsync url... it’s hard to test appsync locally as it runs a custom Apache VTL engine which is hard to recreate. AWS team : appsync local server would be awesome FYI. Besides this you can still write unit tests for all your lambda code and finally for integration testing on lambda, I run an express server with some custom middleware to make inbound requests / events look like api gateway events that lambda receives and also wrap the callback function. I can share this to assist if that helps but let me know as will take some time (will have to make a reproduction of a closed source project) generally for local testing I use the following tooling:
AWS dynamodb local
My own express “lambda” server
AWS step functions local (set it to connect to dynamodb and my own express lambda server)
Local stack for cognito services and S3... warning that lots of localstack features I found don’t work.

The above tooling covers all of my code needs except for appsync Apache vtl templates.... for this as said I write actually graphql query and mutations and my tests run on an AWS sandbox environment... also, I use snapshot tests here to see if my vtl templates ever change on deploy.

Hey @danielblignaut
Thanks so much for your very detailed reply and especially for pointing me towards CDK.
I am still trying to setup a backend where I have several lambda functions which share a common lib (which will ultimately talk to dynamo-db).
I got pretty far with CDK and tried to follow you instructions as best as I could.
I created a very simple demo with one hello-world lambda importing from one common my-lib package.
https://github.com/ziggy6792/aws-cdk-lambda-shared-package
If I deploy this stack to AWS and run my hello-world lambda (by testing from the AWS console) it works! (It successfully imports my-lib does not error).
However once again I have the problem of mocking locally.
I have tried to use this method (which I found here) to mock locally (this method works fine when I don't import my common my-lib).

cdk synth --no-staging > template.yml (to find the logical lambda function id = HelloWorldLambda5A02458E)
sam local invoke HelloWorldLambda5A02458E

But I get an error
{"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'my-lib/MyLib'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js"}

My questions are...

Am I on the right tracks? Is the setup I have close to what you were suggesting? Any improvements I should make?
Do you know how I can test my lambda function locally (not being able to test locally is a deal breaker for me)?

Thanks a lot 👍

@ziggy6792 ,
Some feedback:

At the moment my CDK project is closed source, but I will work to get a minimum reproduction of the lambda proxy, how I test my lambdas using jest, and deploying multi environment CDK projects
For a great introduction to AWS codepipeline with CDK and deploying to multi-env aws accounts see this tutorial https://docs.aws.amazon.com/cdk/latest/guide/cdk_pipeline.html
Mocking a lambda for local development involves re-creating a lightweight web server that can execute the lambda. For now, take a look at https://github.com/localstack/localstack and particularly the lambda documentation (localStack can be buggy though so here's another project to help with lambda mocking that I found from a quick google https://www.npmjs.com/package/lambda-local
a brief tutorial on mocking api gateway lambdas https://steveholgado.com/aws-lambda-local-development/
More important than true lambda mocking for me is having a strong testing environment for automated unit, integration and end to end tests

Here's an example CDK repo I've setup with 2 lambdas, a common library and a CDK project. Includes all the bells and whistles except multi-aws account deployment (point 2) and testing environment could use a lot of work.
https://github.com/danielblignaut/cdk-monorepo-example

@danielblignaut wow this is incredible.
Thank you so much!!!

I found the best way to test the lambda functions locally is to bypass trying to run them as lambdas completely... I know sounds crazy but stick with me!
The key is really adding this to your jest.config.js to redirect the module imports on the js files in your backend folder
moduleNameMapper: {
    '^.+\\.(css|styl|less|sass|scss|png|jpg|ttf|woff|woff2)$':
      'jest-transform-stub',
    '^/opt/(.*)$': '<rootDir>/amplify/backend/function/core/opt/$1', <--This line
  },

I then have a set up function called configEnv() that i call in beforeAll() setup function
import * as config from '../../../aws-exports';
import * as meta from '../../../amplify/backend/amplify-meta.json';
import * as localEnv from '../../../amplify/.config/local-env-info.json';

export const configEnv = () => {
  var credentials = new AWS.SharedIniFileCredentials({profile: 'amplify'});
  AWS.config.credentials = credentials;
  AWS.config.region = config.default.aws_appsync_region;
  process.env.API_MYAPP_GRAPHQLAPIIDOUTPUT = meta.api.myapp.output.GraphQLAPIIdOutput;
  process.env.API_MYAPP_GRAPHQLAPIENDPOINTOUTPUT = config.default.aws_appsync_graphqlEndpoint;
  process.env.ANALYTICS_ROARNOTIFICATIONS_ID = config.default.aws_mobile_analytics_app_id;
  process.env.ANALYTICS_ROARNOTIFICATIONS_REGION = config.default.aws_mobile_analytics_app_region;
  process.env.REGION = config.default.aws_appsync_region;
  process.env.ENV = localEnv.envName;
}

and then I just set up a test:
import {handler} from '../../../amplify/backend/function/someHandler/src/index';

describe("Test some handlers", () => {
  it("Should run some handler", async () => {
    const event = {
      typeName: 'Mutation',
      fieldName: 'someHandler',
      arguments: {
        input: {
          someData: 12335
        }
      },
      identity: {
        username: '<cognitoId>'
      }
    }

    const response = await handler(event);

    expect(response).toBeTruthy();
  })
})

Obviously this will run as integration tests hitting your actual servers so if you need to hit a local stack then change the configEnv().  Personally I think REAL integration tests are highly underrated but I will write a bunch of unit and component tests with the various parts mocked out and then leave one or two full integration tests in to run against a proper stack.  This is great in amplify because you can use a sandbox environment for most stuff.  I don't really like setting up a local stack to "emulate" because they rarely behave like a real environment and I find most time lost on projects is debugging issues with the testing environment rather than the actual code. Since taking this approach and dropping localstack/amplify mock etc. I have saved tons of time in debugging.
This jest module redirect technique also works with layers etc. which the current amplify mock and such don't so this is much much easier.  Also works well with typescript because you don't need to build the functions before you test locally if your jest is setup to support typescript.  You only need your amplify build command to run tsc in packages.json in the root project - "amplify:someHandler": "cd amplify/backend/function/someHandler/src && tsc

@danielblignaut
Some feedback on your project. Firstly thanks so much again. This is by far the best way I have seen to locally mock and test lambdas. My productivity is going to increase so much with this setup.

CDK Pipeline. I did the tutorial.
Now I want to add a pipeline to this project. Would you suggest to put the frontend and backend in the same git repo and the same pipeline?
So the pipeline would be something like;

Commit and push changes
          |
Create dev stage (deploy front end + lambdas + gateway)
          |
Manual approval
          |
Create prod stage (deploy front end  + lambdas + gateway)

I guess the disadvantage of this is every-time I commit a change to my front my whole backend will be redeployed too (and vice versa).
What do you think?

Absolute imports. It seems I can't use absolute imports in my lambdas right? E.g: below image I am trying to import ./example/example using the absolute path (from lambda-a src) and it doesn't work (I also tried to import absolutely from the root packages/lambda-a/src/example/example and that didn't work either). This isn't a huge problem, I just need to remember not to use them.

I tried adding
"compilerOptions": {
     ...
    "baseUrl": "./",
  },

To lambda-a tsconfig.build.json but it didn't help.

@ziggy6792 ,
no problem. Glad it's helping! I've built other tooling around appsync and local environment setup that I'm hoping to publicly share but there's a lot of work documentation that I need to create first. Glad to hear it's helping someone! Your comments:

I'd generally seperate backend and frontend into 2 seperate projects or git repos. Whilst using lerna and monorepo's you can do this, I'd generally advise against it. The main reason for lerna is to be able to improve development efficiency when working on multiple packages that are linked together without having to work on one package, build, deploy it and then switch to another npm package project to work on. Whilst you may have some shared code for your GraphQL schema or something similar, I fear it would run the risk of promoting bad habits by sharing code from backend to front end that generally shouldn't happen. Also your environments are very different, one is browser based and other is server nodeJS. But again, this is personal preference my side. If you would like to be able to re-use the same build pipeline across projects or in general, if you would like to re-use your CDK code, I'd advise looking at Constructs in CDK. These are how they promote creating re-usable higher level components not tied to a specific stack and scope. It may seem natural to just re-use stacks but I've read in a few github issues that constructs are the way to go. Finally, I personally use nextJS and vercel for all frontend REACT.JS projects as the setup is so quick and simple so my experience with deploying frontends on AWS is not that in-depth.
I've never used absolute imports TBH. The only import tooling I use that's not in that project is usage of typescript paths in my test environment so that I don't have "../../../../" to get to the root of the project where I keep some test utils. This can be done by configuring jest and the tsconfig.test.json . This might not be the answer you're looking for but a paths setup may help for what you're looking for.

When I have a chance I'll add my CDK pipeline tools to that project that may help as well.

Awesome thanks 👍

@danielblignaut
I have done quite a lot of work in the foundations for my app running using cdk (setup some lambdas, api gateway, cognito user pool etc.. ) and I have a react frontend which can authenticate with cognito and then call my api.
I have a frontend repo and a backend repo as you suggested and I would now like to setup a pipeline for both; so that changes merged to master are automatically deployed to dev and then prod (after a manual approval step).
It's pretty clear to me how to setup the backend pipeline as it is just an extension of the starter project you already shared. However I am less clear about how to setup the frontend pipeline.
Would you suggest a folder structure like this?
So I would have 2 yarn workspaces

The CDK app that describes the pipeline (deploying my frontend to S3 for dev and prod stages)
The actual frontend

For me the complicated bit is going to be configuring the tsconfig and .eslintrc correctly (as one workspace will have to be setup for react and running in the browser and the other workspace will be a normal node js setup).
I can probably get this working on my own. Just wanted to know if I am on the right tracks here. Or is there a better/simpler solution? Also if you have an example of typescript react frontend that is deployed to S3 using a CDK pipeline that would be amazing.
Thanks so much for your help so far.

This seems like a fairly straight forward feature to add...isn't it just a different template in addition to the ones for nodeJS and python etc...someone just needs to provide the simplest typescript project that works on lambda and add it to the menu I would think?

after seeing many of the approaches here (thanks to all for documenting them!), i’ve landed on a setup using esbuild --bundle and yarn workspaces that minimizes the size of the lambda functions that get deployed. my lambdas all have empty node_modules thanks to yarn workspace dependency hoisting and minimal script sizes thanks to the bundling and tree-shaking via esbuild (total size of my lambdas range between 20KB–300KB including aws-sdk v3 dependencies). here’s the full write-up: https://www.acusti.ca/blog/2022/03/30/writing-your-amplify-functions-in-typescript-via-esbuild-and-yarn-workspaces/
the most important bits are the build script in each amplify function’s package.json, which runs tsc for type-checking then esbuild for transpiling / bundling (see the blog post for details on the esbuild options):
  "scripts": {
    "build": "tsc -noEmit && esbuild *.ts --main-fields=module,main --bundle --platform=node --external:@aws-sdk/signature-v4-crt --outdir=./"
  }

and the per-function build script in the amplify root package.json:
  "scripts": {
    "amplify:<functionName>": "cd amplify/backend/function/<functionName>/src && yarn && yarn build"
  }

i also use baseUrl in my amplify root tsconfig and extend it from the function-specific tsconfigs so that i can easily and cleanly import GraphQL queries and mutations and types (see the blog post for full tsconfig.json contents):
import { GetItemQuery, GetItemQueryVariables } from 'API.js';
import { getItem } from 'graphql/queries.js';

We typically use Webpack to bundle up the solution. Sharing our settings here as well:
Root package.json:
"scripts": {
    "amplify:<functionName>": "cd amplify/backend/function/<functionName> && yarn install && webpack && cd -"
  },

The function's tsconfig.json:
{
  "compilerOptions": {
    "lib": [
      "es2017",
      "DOM"
    ],
    "target": "es2017",
    "module": "commonjs",
    "moduleResolution": "node",
    "baseUrl": "./",
    "sourceMap": true,
    "esModuleInterop": true,
    "resolveJsonModule": true,
    "types": [
      "node",
      "jest"
    ],
  },
  "exclude": [
    "node_modules",
    "**/*.test.ts"
  ]
}

Our webpack.config.ts for the function (note that this file is TypeScript as well):
const path = require('path');
import webpack from 'webpack';

const env = process.env.ENV ? process.env.ENV : "dev"

const config: webpack.Configuration = {
  mode: env === 'main' ? 'production' : 'development',
  // devtool: 'source-map',
  entry: './lib/index.ts',
  target: 'node',
  node: {
    __dirname: true,
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },
  resolve: {
    extensions: [ '.tsx', '.ts', '.js' ],
    modules: ["node_modules"]
  },
  output: {
    filename: 'index.js',
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, 'src'),
  }
};

export default config

The source files live in the /lib directory in the function's folder, while the compiled output is placed into /src.

+1

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

It works just as I described. Develop your backend code as an independent npm package, that is installed via a lambda layer with this package in the layer's package.json.

@r0zar has a point.
I personally develop all my backend code as an independent typescript node package. You can then use all the things you like such as jest for unit testing. The lambda then installs that private node package and becomes a very thin javascript layer on top of that node package built in typescript.
This has worked out well and lets us develop and test things independently of the amplify workflow. The package also becomes very re-usable for scenarios outside of amplify (such as our own cli to interact with our backend).

That's an interesting approach. How does it work?

It works just as I described. Develop your backend code as an independent npm package, that is installed via a lambda layer with this package in the layer's package.json.

Oh, right, I missed it was "private node package". Makes sense.


Add ocm user role

https://issues.redhat.com/browse/SDA-4989

@vkareh @zgalor @ciaranRoche  PTAL

@vkareh changed it to use to the ocm account name

Thanks @pvasant!!
/lgtm


Example breaks windows testing

#263 has enable doctests and as a result the appveyor build is complaining file access error as it's being used by another process. (The weird thing is that all tests pass, so it would be nice to be able to turn this strictness off).
The workaround I ended up with is to allow the failures of the appveyor jobs to let the release procedure for 0.4 proceed.
cc @pllim @astrofrog @saimn as you are the ones who solved similar issues in astropy before.
============= 955 passed, 8 skipped, 13 xfailed in 35.62 seconds ==============
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\conda\envs\test\lib\site-packages\astropy\utils\decorators.py", line 860, in test
    func = make_function_with_signature(func, name=name, **wrapped_args)
  File "C:\conda\envs\test\lib\site-packages\astropy\tests\runner.py", line 260, in test
    return runner.run_tests(**kwargs)
  File "C:\conda\envs\test\lib\site-packages\astropy\tests\runner.py", line 605, in run_tests
    return super().run_tests(**kwargs)
  File "C:\conda\envs\test\lib\site-packages\astropy\tests\runner.py", line 242, in run_tests
    return pytest.main(args=args, plugins=plugins)
  File "C:\conda\envs\test\lib\site-packages\astropy\config\paths.py", line 182, in __exit__
    shutil.rmtree(self._path)
  File "C:\conda\envs\test\lib\shutil.py", line 513, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\conda\envs\test\lib\shutil.py", line 392, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\conda\envs\test\lib\shutil.py", line 392, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\conda\envs\test\lib\shutil.py", line 392, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\conda\envs\test\lib\shutil.py", line 397, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\conda\envs\test\lib\shutil.py", line 395, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\tmppexf4t80astropy_cache\\astropy\\download\\py3\\2c9202ae878ecfcb60878ceb63837f5f'

I remember seeing it but I don't remember any solution. It is how Windows deals with the file pointer differently (insert mystery music), causing this message. If others remember a solution, please chime in.

I found something here -- not sure if it is a solution for you, but apparently Past Me ™ fixed it by caching.
https://github.com/astropy/astropy/issues/7757#issuecomment-415592313

Also maybe try memmap=False if it is FITS file. xref astropy/astropy#7404

Yes, I've tried the solution from astropy/astropy#7757, it had no effect (haven't tried memmap). Do you also think it's the fits files that cause the problem?
Honestly I run out of steam of sorting windows out, anyone is more than welcome to grab this and run with it.

If I run the tests twice I now get this error:
_______________________________________________________________________ [doctest] regions.io.fits.write.write_fits_region _______________________________________________________________________
063     Examples
064     --------
065
066     >>> from astropy.utils.data import get_pkg_data_filename
067     >>> from astropy.io import fits
068     >>> file_sample = get_pkg_data_filename('data/fits_region.fits', package='regions.io.fits.tests')
069     >>> from regions import CirclePixelRegion, PixCoord, write_fits_region
070     >>> reg_pixel = CirclePixelRegion(PixCoord(1, 2), 5)
071     >>> hdul = fits.open(file_sample)
072     >>> write_fits_region('region_output.fits', regions=[reg_pixel], header=hdul[1].header)
UNEXPECTED EXCEPTION: OSError("File 'region_output.fits' already exists.")
Traceback (most recent call last):

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/doctest.py", line 1329, in __run
    compileflags, 1), test.globs)

  File "<doctest regions.io.fits.write.write_fits_region[6]>", line 1, in <module>

  File "/Users/deil/work/code/regions/regions/io/fits/write.py", line 78, in write_fits_region
    bin_table.writeto(filename)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/utils/decorators.py", line 521, in wrapper
    return function(*args, **kwargs)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/io/fits/hdu/base.py", line 1543, in writeto
    checksum=checksum)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/utils/decorators.py", line 521, in wrapper
    return function(*args, **kwargs)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/io/fits/hdu/hdulist.py", line 919, in writeto
    fileobj = _File(fileobj, mode=mode, overwrite=overwrite)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/utils/decorators.py", line 521, in wrapper
    return function(*args, **kwargs)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/io/fits/file.py", line 178, in __init__
    self._open_filename(fileobj, mode, overwrite)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/io/fits/file.py", line 544, in _open_filename
    self._overwrite_existing(overwrite, None, True)

  File "/Users/deil/software/anaconda3/envs/regions/lib/python3.7/site-packages/astropy/io/fits/file.py", line 438, in _overwrite_existing
    raise OSError("File {!r} already exists.".format(self.name))

OSError: File 'region_output.fits' already exists.

/Users/deil/work/code/regions/regions/io/fits/write.py:72: UnexpectedException

The problem is that that doctest writes to the current working directory.
That's no good.
Personally I don't like doctests at all, and my suggestion would be to simply skip that example and not run it as a doctest. But probably another course of action could be to try and configure it somehow to run in a temp folder?
@pllim @bsipocz - Thoughts?

Personally I don't like doctests at all, and my suggestion would be to simply skip that example and not run it as a doctest. But probably another course of action could be to try and configure it somehow to run in a temp folder?

Personal dislikes here I don't think count. We shouldn't have examples in the docs that are not guaranteed to run. I would  be much more comfortable removing the examples than skipping testing them.
So I agree that the test should be fixed either to write to a temp file, or to remove the writing part of it. Sadly I don't have time to work on regions in the near future.

There's three doctests that write to an output file in the CWD, and that fail when one runs the tests via pytest regions:
	region_output.fits
	test_write.crtf
	test_write.reg

@pllim - I'll wait to see what you come up with in #268 , but probably that's unrelated, and then I could make a PR which fixes the doctests somehow. What is a good way? import tempfile in examples users read is not great. Is there a better way?

@cdeil , why not add an overwrite option to the API instead?
https://github.com/astropy/regions/blob/66ab0cdf40be23b2c72b23bcfbe64a9a06db9452/regions/io/fits/write.py#L50
Your problem here will not be solved with my PR to debug Windows file error.

@cdeil - why did you reopen this? Appveyor is all green now.

@bsipocz - I re-opened because I wanted to resolve the remaining issue discussed here. Posted my suggestion in #273 .

We shouldn't have examples in the docs that are not guaranteed to run. I would be much more comfortable removing the examples than skipping testing them.

In #273 I removed them. I'm not sure why the equivalent docs examples in the docs folder don't create temp files in the CWD. Maybe they aren't picked up?
I really think insisting to run all examples as tests is too much. Good examples don't make for good tests, and IMO putting import tempfile isn't appropriate for end-user examples. See e.g. http://docs.astropy.org/en/stable/table/io.html - you'll find a doctest-skip-all at the top, and similarly in many places where in the Astropy docs examples files are written, those examples are not executed as doctests.

yes, and we're having issues from time to time that those examples are breaking which is both embarrassing and stressful. It's rather a nightmare really to keep on top of the examples working as they suppose to work if they are not tested. I agree that they are not good unit tests, but excellent integration tests 😉

(And tests in http://docs.astropy.org/en/stable/table/io.html  arguably are skipped (just as in the whole of astroquery) as we don't have a crucial piece of infrastructure to support better setup for examples that require remote data access, see the long standing issue of https://github.com/astropy/pytest-doctestplus/issues/2 and it's precursors)

Disagree that doctests are good integration tests. A good test always has asserts at the end. A docs example never has asserts at the end. Sometimes you can get away by printing something, but in the case of I/O functions like here, the deficiencies of doctests really become clear.
I do agree that executing docs examples is valuable to have easier-to-maintain docs. But when it's not possible to have both (good simple example, and doctest), I prefer to keep the good simple example and to not have it tested.

Well, my PR showed that many of these simple examples were broken, and many with Syntax Error. While I hear your point about simple examples, I think for a multi people project all examples needs to be tested to keep things consistent.


Add normalized name and publisher to index and correlation telemetry

Change
This change adds the normalized name and publisher to the index, as well as leveraging this to collect telemetry on how well this normalization strategy is doing to correlate a manifest with the ARP entry on the machine.
The normalized name and publisher tables are added to the index as 1:N entries due to the introduction of full localization of name and publisher into the new manifest schema.  These tables don't have any link between them, which would allow a combination of name and publisher from any localization to work.  This tradeoff of precision seems worthwhile over writing more complex code to handle two values at a time, and it also allows the potential for one of the fields to change while the other remains constant and consume less space overall.
Searching is done by sending in the observed values for name and publisher, and the normalization is done by the source before comparing with its internal store.  In this way any type of normalization can occur internal to the source without the workflow logic needing to be aware.
In order to collect telemetry, we first snapshot the set of packages in ARP before executing the install.  Afterward, we compare the set with the snapshot to determine which ones are net new.  We also search the source using the manifest used to install to determine if there are any matches.  Finally, we intersect these two sets of packages.
The telemetry contains information about the package that was being installed {source, id, version, channel}, information about the number of packages in each state above {changed, matching, intersection}, and information from the ARP entry (if any) that we have decided is the one that represents the main package {name, version, publisher, language}.  We will be able to leverage this telemetry to improve both our normalization code and the manifests in winget-pkgs to improve our ability to correlate installed packages with remote ones.
For instance, installing Notepad++ show the following:
[CLI ] During package install, 1 changes to ARP were observed, 0 matches were found for the package, and 0 packages were in both
[CLI ] The entry determined to be associated with the package is 'Notepad++ (64-bit x64)', with publisher 'Notepad++ Team'

Notepad++ does not correlate with the manifest because its publisher does not match (note the lack of “Team” in the manifest):
PS> wingetdev show Notepad++.Notepad++
Found Notepad++ [Notepad++.Notepad++]
Version: 7.9.3
Publisher: Notepad++

Validation
New tests are added for the index changes and to verify behavior is consistent with expectations with the ARP reporting.
Microsoft Reviewers: Open in CodeFlow

Fwiw, I'd suggest adding \bE2EPackaged\b on its own line to patterns.txt instead of adding the suggested item to expect.txt.


Rename mdast-contributors to remark-contributors

Rename mdast* to remark*;
Add mdast-util-to-string as a dependency, and use it to
get the value of heading nodes, which results in faster and
simpler stringification;
Add wooorm to the table of contributors.

You’ll still need to:

Rename the project on GitHub to remark-contributors;
Publish remark-contributors to npm;
Preferably update mdast-contributors on npm to show a deprecation message.

I’d be up for doing the last two for you if you’d give me access to npm :)
P.S. Read more about the changes here.

Oops, sorry for the delay @wooorm! Thanks for this — adding you on GH/npm and will take a look at making those final changes :D

Thanks @hughsk! And not to worry :+1:
I missed some replaces in the tests, I’ll whip  up a new release. Could you update the GH repo description for remark too?

Sure thing! Done :)

Cool! :ok_hand: Just pushed f54f168b with the final replaces, found a bug in remark so that took a little bit longer.
Let me know if you want to release this commit. Don’t think there’s a point as the changes are in .npmignored files though...

Yeah, since the files are ignored should all be good :) Thanks!


NISO STS import on ISO Fonto: cross-references in terminology and bibliography entries not working

As reported by SIS (sis.se). Have requested for more information.
This is probably due to ISO Fonto implementing terms and bibliographic items not as part of the document but in their own individual databases for linking with existing content.

From @matsahlin:
The cross reference in terms I identified from Metanorma as:

But when I recreated what would been what Fonto expected I got this:

The cross references for the Bibliography I found from in Metanorma to be:

But after import to Fonto it looks like this:

And when recreating the cross reference in Fonto I get this:

@Intelligent2013 not sure if there's something we can do but could you have a look? Thanks.

Regarding the issue:

The cross reference in terms I identified from Metanorma as:

I've updated mnconvert to generate tbx:entailedTerm instead of named-content for terms within the Terms and definitions clause as noted in '9.3.6 Cross-references between terms in the Terms and definitions clause' in 'IEC/ISO Coding Guidelines':
<tbx:definition><tbx:entailedTerm target="term_3.1.1">entity (3.1.1)</tbx:entailedTerm> with substantial knowledge, experience and competency in the domain</tbx:definition>

But SIS expects the tag xref instead of tbx:entailedTerm:

But when I recreated what would been what Fonto expected I got this:

If we need to update mnconvert for support SIS practice, then I need the full XML SIS example to adapt Metanorma to STS XSLT conversion rules specially for SIS format.

The cross references for the Bibliography issue fixed. Now it encodes as:
<xref rid="biblref_13" ref-type="bibr">ISO 19115-1:2014</xref>


[Feature] Ctrl+Enter to submit post

Nice shortcut, adopted by various web apps.

It's on the roadmap. PRs welcome!

Hey @jgdovin @dandv, this feature was on Telescope Legacy (blaze) but doesn't exist anymore with the React-based version: I opened a new issue with a step by step guide to do it, if you are interested : #1471 👍


ValueError: Operands could not be broadcast together with shapes (56, 56, 32) (112, 112, 32)

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 2.4.1
Python version: 3.8
Bazel version (if compiling from source):
GPU model and memory: Nvidia GeForce RTX 2080

Describe the problem.
I am trying to build a model with keras using the Functional API. However, I get an error relating to the Add() function of my residual blocks.
Describe the current behavior.
The current error I get is: ValueError: Operands could not be broadcast together with shapes (56, 56, 32) (112, 112, 32)
Standalone code to reproduce the issue.
visible = Input(shape=(224, 224, 3))
x = Conv2D(32, (3, 3), padding='same', name='conv_1')(visible)
x = LeakyReLU(alpha=0.1)(x)
x = RandomFlip(mode='horizontal', seed=None)(x)
x = BatchNormalization()(x)
block_1 = MaxPool2D(pool_size=(2, 2), padding='same')(x)

x = Conv2D(32, (3, 3), padding='same', name='conv_2')(block_1)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_2 = Add()([x, block_1])

x = Conv2D(64, (3, 3), padding='same', name='conv_3')(block_2)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_3 = Add([x, block_2])

x = Conv2D(64, (3, 3), padding='same', name='conv_4')(block_3)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_4 = Add([x, block_3])

x = Conv2D(64, (3, 3), padding='same', name='conv_5')(block_4)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_5 = Add([x, block_4])

x = Conv2D(64, (3, 3), padding='same', name='conv_6')(block_5)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_6 = Add([x, block_5])

x = Conv2D(128, (3, 3), padding='same', name='conv_7')(block_6)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_7 = Add([x, block_6])

x = Conv2D(128, (3, 3), padding='same', name='conv_8')(block_7)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_8 = Add([x, block_7])

x = Conv2D(128, (3, 3), padding='same', name='conv_9')(block_8)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_9 = Add([x, block_8])

x = Conv2D(128, (3, 3), padding='same', name='conv_10')(block_9)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_10 = Add([x, block_9])

x = Conv2D(256, (3, 3), padding='same', name='conv_11')(block_10)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_11 = Add([x, block_10])

x = Conv2D(256, (3, 3), padding='same', name='conv_12')(block_11)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_12 = Add([x, block_11])

x = Conv2D(256, (3, 3), padding='same', name='conv_13')(block_12)
x = LeakyReLU(alpha=0.1)(x)
x = BatchNormalization()(x)
x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
block_13 = Add([x, block_12])

flat_layer = Flatten()(block_13)
x = Dropout(0.35)(flat_layer)
x = Dense(512, kernel_regularizer=regularizers.l2(0.1))(x)
x = LeakyReLU(alpha=0.1)(x)
x = Dropout(0.35)(x)

outputs = Dense(4)(x)

model = Model(visible, outputs, name='Functional_Model')

model.summary()

Source code / logs.
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/home/lunet/conm/Desktop/Stenosis-Project/functional.ipynb Cell 2 in <cell line: 12>()
     10 x = BatchNormalization()(x)
     11 x = MaxPool2D(pool_size=(2, 2), padding='same')(x)
---> 12 block_2 = Add()([x, block_1])
     14 x = Conv2D(64, (3, 3), padding='same', name='conv_3')(block_2)
     15 x = LeakyReLU(alpha=0.1)(x)

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:951, in Layer.__call__(self, *args, **kwargs)
    945 # Functional Model construction mode is invoked when `Layer`s are called on
    946 # symbolic `KerasTensor`s, i.e.:
    947 # >> inputs = tf.keras.Input(10)
    948 # >> outputs = MyLayer()(inputs)  # Functional construction mode.
    949 # >> model = tf.keras.Model(inputs, outputs)
    950 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
--> 951   return self._functional_construction_call(inputs, args, kwargs,
    952                                             input_list)
    954 # Maintains info about the `Layer.call` stack.
    955 call_context = base_layer_utils.call_context()

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:1090, in Layer._functional_construction_call(self, inputs, args, kwargs, input_list)
   1086 if keras_tensor.keras_tensors_enabled():
   1087   with call_context.enter(
   1088       layer=self, inputs=inputs, build_graph=True, training=training_value):
   1089     # Check input assumptions set after layer building, e.g. input shape.
-> 1090     outputs = self._keras_tensor_symbolic_call(
   1091         inputs, input_masks, args, kwargs)
   1093     if outputs is None:
   1094       raise ValueError('A layer\'s `call` method should return a '
   1095                        'Tensor or a list of Tensors, not None '
   1096                        '(layer: ' + self.name + ').')

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:822, in Layer._keras_tensor_symbolic_call(self, inputs, input_masks, args, kwargs)
    820   return nest.map_structure(keras_tensor.KerasTensor, output_signature)
    821 else:
--> 822   return self._infer_output_signature(inputs, args, kwargs, input_masks)

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:862, in Layer._infer_output_signature(self, inputs, args, kwargs, input_masks)
    856 with backend.name_scope(self._name_scope()):
    857   with autocast_variable.enable_auto_cast_variables(
    858       self._compute_dtype_object):
    859     # Build layer if applicable (if the `build` method has been
    860     # overridden).
    861     # TODO(kaftan): do we maybe_build here, or have we already done it?
--> 862     self._maybe_build(inputs)
    863     outputs = call_fn(inputs, *args, **kwargs)
    865   self._handle_activity_regularization(inputs, outputs)

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:2710, in Layer._maybe_build(self, inputs)
   2705 if not hasattr(self.build, '_is_default'):
   2706   # Any setup work performed only once should happen in an `init_scope`
   2707   # to avoid creating symbolic Tensors that will later pollute any eager
   2708   # operations.
   2709   with tf_utils.maybe_init_scope(self):
-> 2710     self.build(input_shapes)  # pylint:disable=not-callable
   2711 # We must set also ensure that the layer is marked as built, and the build
   2712 # shape is stored since user defined build functions may not be calling
   2713 # `super.build()`
   2714 Layer.build(self, input_shapes)

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/utils/tf_utils.py:272, in shape_type_conversion.<locals>.wrapper(instance, input_shape)
    270 if input_shape is not None:
    271   input_shape = convert_shapes(input_shape, to_tuples=True)
--> 272 output_shape = fn(instance, input_shape)
    273 # Return shapes from `fn` as TensorShapes.
    274 if output_shape is not None:

File ~/.conda/envs/stenosis/lib/python3.8/site-packages/tensorflow/python/keras/layers/merge.py:112, in _Merge.build(self, input_shape)
    110   else:
    111     shape = input_shape[i][1:]
--> 112   output_shape = self._compute_elemwise_op_output_shape(output_shape, shape)
    113 # If the inputs have different ranks, we have to reshape them
    114 # to make them broadcastable.
    115 if None not in input_shape and len(set(map(len, input_shape))) == 1:
...
     85           'together with shapes ' + str(shape1) + ' ' + str(shape2))
     86     output_shape.append(i)
     87 return tuple(output_shape)

ValueError: Operands could not be broadcast together with shapes (56, 56, 32) (112, 112, 32)

I solved this problem by removing the maxPool layers as this was making my data shape smaller each time.

@NevilleMthw Thank you for the update!


Preroll does not work on newest version on mobile browser

Hi,
I am using latest version of videojs-ima and got issue with mobile browser(tested with safari, chrome, firefox).
Even using the sample pages: https://googleads.github.io/videojs-ima/
Error from console when started playing video:

Ad error: AdError 400: There was an error playing the video ad. Caused by: AdError 1205: The browser prevented playback initiated without user interaction.

The only one works well is the autoplay sample which has "muted playsinline" defined, so on mobile do we need to set "muted playsinline" on video html default to prevent this error?

hi @ypavlotsky I found the solution, by adding playsinline attribute into the video element, it works well on mobile browser!

I'm still seeing the same error with playsinline. What are you testing on, in terms of device and OS?

@ypavlotsky I am testing with all OS including iOS, Android(Safari, Chrome, Firefox).
May be try to run your test with muted playsinline attribute?

@ypavlotsky
Update: Sometime, preroll video still stopped due to the warning about user interaction with the implementation using playsinline, muted

+1

Update: Mobile Preroll works on vjs 5.20.4, vjs contrib ads 5.1.6 and vjs ima 1.1.1. Tested with Safari, Chrome, Firefore on iOS, Android.
Solution is:
On mobile mute the player and provide setting adWillPlayMuted + mute the ads before request preroll vasttag with: this.player.ima.requestAds();

Still haven't heard back from the SDK team on this issue, will update here when I have more information.

Right after error 1205 appears, another error comes out
Uncaught TypeError: Cannot read property 'v' of null
at nA (bridge3.202.0_en.html:427)
at bridge3.202.0_en.html:501
http://imasdk.googleapis.com/js/core/bridge3.202.0_en.html

some mobile browsers requires user interaction to load or play media

touchstart touchend touchmove touchleave touchcancel click keydown keyup keypress

I had used other button to play the audio
$('.player-button').click(function () {
    playerId.player.play();
}

to solve the problem I call the click of the player button videojs by code and listen the click event "this is a user interaction"
$('.player-button').click(function () {
    $('.playerContainer .vjs-big-play-button')[0].click();
}

this.player.one('click', function () {
      _this.player.ima.setContentWithAdTag(null, _this.adUrl, false);
      _this.player.ima.requestAds();
      _this.player.ima.initializeAdDisplayContainer();
      _this.start = true;
    });

this just worked calling the initializeAdDisplayContainer method at the end

This should be fixed with this PR now. If it's still happening, feel free to reopen.


Added Note on Transactional Replication and Distributed AGs

Distributed AG Forwarders cannot be setup as a publisher in transactional replication in SQL16/17. This was confirmed by the product group but not in the documentation. Added a note expressly calling this out. I believe this is one of the better places for the information as the Distributed AG information is just a single page and doesn't' have a "non-supported" features area.

@SeanGallardy : Thanks for your contribution! The author, @allanhirt, has been notified to review your proposed change.

I'm fine with this change.


Missing metrics when querying with recording rule

Thanos, Prometheus and Golang version used:
docker.io/bitnami/thanos:0.31.0-scratch-r12

Object Storage Provider:
GCS
What happened:
We recently started seeing missing metrics when a recording rule is used to query. When the original query expression associated with the recording rule is used, there are no missing metrics at all:
(Top graph is using recording rule, bottom graph is using normal query expression)

We originally didn't have Thanos Ruler enabled, but we were somehow still able to query using recording rules configured for Prometheus. So we thought it might be because Thanos Ruler is not configured, so we then enabled Ruler to see if it solves the problem. However, the problem stayed the same when Ruler was configured, it was surely the Thanos Ruler that evaluated the recording rule, but there were still missing metrics. In the following picture we used a new recording rule configured for Thanos Ruler, that maps to the same expression as the original problematic recording rule:

Some interesting observations:

After a few days, we requeried the same time range as the first picture, using the same recording rule. Interestingly, there were no longer missing queries for that time range. However, this is not always the case, as in many cases we can still see missing metrics for other old time ranges.
Missing time range seem to correspond to the data blocks of different compaction levels (like 2h, 8h):

What you expected to happen:
Metrics returned from query using recording rule has no missing data blocks.
How to reproduce it (as minimally and precisely as possible):
Have Thanos Ruler enabled with a recording rule, writing to a remote storage, and have Thanos Query, Store, Compactor configured.
Full logs to relevant components:

Anything else we need to know:

Can you upgrade to 0.32.5 and retry; 0.31 has some deduplication issues.

Thank you for the fast response, will try the upgrade to 0.32.5.

Upgrading to 0.32.5 seems to solve the issue, thank you for the help!

MichaHoffmann

Upgrading to 0.32.5 seems to solve the issue, thank you for the help!

@MichaHoffmann Yes the issue can be closed :)


I'm faced with an interesting auto-scaling question

I have the solution in place to distribute computations to containers using a Service Bus queue and have HPA setup to scale on the metric below.  It works very well, thanks very much for making it possible!
metric:
metricName: ActiveMessages
aggregation: Total
filter: EntityName eq 'requests'
The scenario that I'm faced with is that most computations complete under 2' while some take 40'.  Their ratio seems to be 22:1 in favor of the short ones.  The short requests come in sporadically, there is no way to predict when and how many will arrive.
The problem is that some of the long computations are ended prematurely - most likely because HPA terminates the container - causing them to be resumed on a different container, which makes their execution even longer.  The customer is not happy with that!
Does anyone has some suggestions on how should I go about ensuring that the long computations complete on their first attempt?  I searched and did not see any way for the container to tell HPA "leave me alone, I'm working".   If I add CPU usage as a secondary scaling criteria, won't that drive the count of containers to max, even if it is only one single long running computation that's executing?
A different solution I imagine is to save the computation's state from time to time and whenever it is forced to resume in a different container, to start from the saved state.  It's not a component I'm working on, so do not know if it is even possible.

This is actually a common challenge that we have run into as well.  KEDA documents some of the solutions for long running executions as well as provides job based scaling option.  We have also implemented the ability to "checkpoint" the computational state as you pointed out in some cases.
Hope this helps.

Thanks very much James!  Is KEDA “production quality” please?  Also, how quick is KEDA to react to new messages in an Azure Service Bus queue?


Sample instructions to execute searches has a bug ;-)

Thanks for the script! I believe this command sample:
python vaccine.py python vaccine.py --state NJ --cities princeton plainsboro
should be this:
python vaccine.py --state NJ --cities princeton plainsboro
...at least that got it to work for me on MacOS.
thanks!

Noticed the script issue as well. Thanks for pointing it out!


Publish a draft roadmap and project goals

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
It would be nice to get a feel about the goals of this projects as well as add some roadmap items. I think it would be massively valuable for folks like me who work in this domain, are willing to contribute and can consider this for early adoption.
Describe the solution you'd like
A one pager would suffice.

Great idea indeed and started a work on it:
https://github.com/kf5i/k3ai-site/issues/3
https://github.com/kf5i/k3ai-site/issues/4
I'll open today a PR and link you...as soon I'll have a draft I'll tag you so you may share your feedback :)
Thanks for sharing your feedback that's super important for us!

Hi @harsimranmaan  we released:

roadmap: https://github.com/orgs/kf5i/projects/2
goals: https://docs.k3ai.in/what-we-try-solve-a.k.a-our-goals

The roadmap is still a work in progress since we have to add more issue there..but please let us know the feedback you have on this..

Thanks you so much @alfsuse


Explorer not showing files when the folder is opened via terminal

Type: Bug
VS Code Explorer is NOT showing the folder contents (files such as *.html) when opened via Mac OS Terminal using the "code ." command. At the sametime all the files are visible when the same folder is opened via the VS Code GUI.
VS Code version: Code 1.74.2 (e8a3071ea4344d9d48ef8a4df2c097372b0c5161, 2022-12-20T10:32:21.587Z)
OS version: Darwin arm64 21.6.0
Modes:
Sandboxed: No

System Info

Item
Value

CPUs
Apple M1 (8 x 24)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: disabled_off

Load (avg)
2, 2, 2

Memory (System)
16.00GB (0.09GB free)

Process Argv
--crash-reporter-id 80432c20-ca8e-4aea-95b8-2fb1c4784ff9

Screen Reader
no

VM
0%

Extensions (8)

Extension
Author (truncated)
Version

isort
ms-
2022.8.0

python
ms-
2022.20.1

vscode-pylance
ms-
2022.12.20

jupyter
ms-
2022.11.1003412109

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.12

vscode-jupyter-cell-tags
ms-
0.1.6

vscode-jupyter-slideshow
ms-
0.1.5

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256859
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscorecescf:30445987
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593cf:30376535
pythonvs932:30410667
cppdebug:30492333
vsclangdf:30486550
c4g48928:30535728
dsvsc012cf:30540253
azure-dev_surveyone:30548225
pyindex848:30577860
nodejswelcome1:30587005
2e4cg342:30602488
iaj6b796:30613358
89544117:30613380
fim-prod:30623723

\gifPlease

Thanks for reporting this issue! Unfortunately, it's hard for us to understand what issue you're seeing. Please help us out by providing a screen recording showing exactly what isn't working as expected. While we can work with most standard formats, .gif files are preferred as they are displayed inline on GitHub. You may find https://gifcap.dev helpful as a browser-based gif recording tool.
If the issue depends on keyboard input, you can help us by enabling screencast mode for the recording (Developer: Toggle Screencast Mode in the command palette). Lastly, please attach this file via the GitHub web interface as emailed responses will strip files out from the issue.
Happy coding!

The issue was with the parent directory. I have made it a trusted source and things are looking fine now. Thank you for the help.


Spirit Blossom Thresh Human Form

If you can fix spirit blossom thresh he has a human form you can access using ctrl+5.

#1


Exponential operator in if condition is not transpiled

Original source code:
if (2 ** 3 > 0) { alert('test'); }

Expected transpiled code:
0 < Math.pow(2, 3) && alert("test");

Actual result
0 < 2 ** 3 && alert("test");

Same result in ternary operator.

Also, it throws an error in Advanced Optimization.
JSC_INTERNAL_ERROR_UNEXPECTED_TOKEN: Internal Error: TypeCheck doesn't know how to handle EXPONENT at line 1 character 4
if (2 ** 3 > 0) { alert('test'); }
    ^

If another exponential operator is in body, they are transpiled.
Original:
if (2 ** 3 > 0) { alert(2 ** 3); }

Transpiled:
0 < Math.pow(2, 3) && alert(Math.pow(2, 3));

I can't seem to repro this with the compiler jar so this seems to be a quirk of the webservice. Will take a look.

@MatrixFrog see this repo for repro https://github.com/teppeis-sandbox/closure-compiler-repro-2821

@teppeis I've reproduced this in the unit test. I'm debugging it now.

Even simpler repro
(2 ** 3) / 3;

Looks like the root cause is in IRFactory.
It fails to record that the exponent operator is used in some cases.

correction to repro
2 ** 3 > 3

fix sent for internal review

@MatrixFrog @brad4d thanks!


[Production & Staging] Importing Github access token gives 500 error

Steps

Go to a project you have admin privileges on
Open the javascript error console for the current browser tab
Go to the settings tab and import the access token for the Github add-on
Select a private repo and hit Submit

Expected
I would expect the private repo to be linked to the project successfully
Actual
There is an error message 'Settings not updated'
And a console error of 500 internal server error

Linking this issue also: https://github.com/CenterForOpenScience/osf.io/issues/2603#issuecomment-103205734
And @icereval and @chrisseto as I showed them the error and they looked into the sentry logs of the event.

Would I be able to work on this?

OK -- see if you can replicate it first @PatrickEGorman

I seem to be getting a 404 error instead.  The repo is failing to link to the project

I realized that my repo was empty.  Also, I'm thinking you need a private repo in github which I believe costs money.  I think I'll have to find a new issue.

@PatrickEGorman if you are using a edu email for github registration and verify you are a student, you will enjoy the right to create a free private repo.

I applied for the free private repo but have yet to hear back.  I hope to hear back and start work by or before Monday.

I have yet to hear back from github.  I double checked and I definitely made the request.  I might suggest passing this issue to somebody who has permissions to create a private repo

@nicipfeiffer @lbanner I'm able to successfully connect my private repos. Maybe this isn't an issue anymore?

I don't think this is a problem anymore. I did link to my private github repo on production and staging.


fix ODS no feature generation bug

In this commit, some bugs are fixed:1) null pointer exceptions and 2)P4J features for multiple changed files.

Hi @SophieHYe
Thanks a lot for the bug fixing.
Do you have a test cases with the case that exposes the bug and assert that is not present any more?
Thanks
Matias

Hi @martinezmatias,
Thanks for reminding. I have updated the existing test case, where the previous result of Defects4J Math-42 human patch was UNKNOWN due to the bug of feature extraction. Since the bug has fixed, the result now for this patch has updated to CORRECT.

Thanks @SophieHYe, added just two small comments.

great @SophieHYe !


export all types from index.ts

This exports all of the types, a very important step to generating index.d.ts. It supplants #3520 and is a step on the path to #3500

ah, need the config too. Closing for now


Possible to select multiple values and form an object?

Hey,
I have a context object that contains multiple things:
export type MyStoreType =
{
    currentColor: string;
    currentTool: "pan" | "pen" | "eraser";
    currentView: number;
}

export const MyStoreContext = createContext<MyStoreType>(...)

Actually, the MyStoreType is much more complex than the example above.
Now in a consumer component, I want to use 'currentColor' and 'currentView' property, and I don't want re-renders caused by the 'currentTool' property. I tried to wrap a more advanced selector, this selector looks like this:
/** Use this hook, get all the values of the context. */
function useHook(): Readonly<MyStoreType & { Update: /* something else */}>
/** Use a selector function to select a part of the context. */
function useHook<TSelectedObject>(selector: (value: TFullValue) => TSelectedObject): Readonly<TSelectedObject> & { Update: /* something else */}>
/** Use an array of properties to select a part of the context. */
function useHook<U extends keyof TFullValue>(selections: Array<U>): Pick<TFullValue, U> & { Update: /* something else */}>

function useHook(selector?: any)
{
    const selectFunc = useCallback((v: MyStoreType>
        | { v: { current: MyStoreType; }}) =>
    {
        const value = "v" in v ? v.v.current : v;

        if (selector === undefined)
        {
            return value;
        }
        else
        {
            let selection = { };

            if (typeof selector === "function")
            {
                selection = selector(value);
            }
            else if (Array.isArray(selector))
            {
                for (const key of selector)
                {
                    selection[key] = value[key];
                }
            }

            return {
                ...selection,
                Update: /* Some additional stuff */
            }
        }

    }, [selector]);

    const value = useContextSelector(context, selectFunc);
    return value;
}

However when I pass an array to this hook, a infinite-loop exception will be thrown.
const editor = useMyStoreContext(["currentView", "currentColor"])

Can the use above be implemented in this way, or another way?
Thanks for ur time!

I think this might be related to #19 (i guess?)

Yes. The current solution is to use two hooks or use memoization library.
(Zustand's useShallow may work too.)


fix(dialog): add panelClass from position to the overlay

CdkConnectedOverlay directive was ignoring panelClass from
cdkConnectedOverlayPositions. This change adds support to add panelClass
from positions to the overlay.

Sorry, left "fit" in code. Closing


Update build.xml to allow for jar compilation

Added the jar target in the build.xml, so that the project may be packaged for use. Includes README, and license for license compliance.

👍
Thanks!


Crash, kind of

So I am playing the Allied Kingdom Pack on the curse launcher. I really want to build with Schematica so I got a schematic and loaded it up . Nothing bad happened the schematic loaded up just fine but the instant I walk into it my minecraft closes. I tried it over and over again even with other schematics. It does not technically crash my game because there is no crash report but my client always closes no matter what can u please help me.

If there are no crash report, there might be either an hs_error_pidXXXX or replay_pidXXXX. Post these errors with your fml-client-latest. I think this can help finding the bug.

Heres what I found I put it in a pastebin.  http://pastebin.com/f2Mt4ZS0

Sorry, I haven't the knowledge to understand this type of error logs. However, logs may help, we need logs\fml-client-latest in order to know what happened and probably what caused that error.

its encrypted do u know how to change it to a readable document.

Encrypted like :
À°’šÖ_Åê%Ô{(—…Æ£Þ º—Ùj`mkZ×ÏŸ>|ùóçÍkXb¦›ý\"únÉµß*L©ÒtÕ>|ùÍw7G|æç'ú»ÁóçÏŸ>|ùóçÏŸ>|ùóçÏŸ>|ùóçÏ…à@‹18ŸµüêÒRHZÃyn$ùxW0–.$pÍFŒ2J%ÀZrgu=þ¯Øà¹ Â«>…3!KÕµJ˜¼ý»+¸†‚oõéh\ðŠæVÍ˜œOÚþu bµk_>|ùóçÏž5i€
?
That shouldn't be encrypted. What is the extension of the file?
In my side, this is .log

mine is a .log too

http://pastebin.com/Z4EprqPC

I'd delete the logs and crash-logs folders and run Minecraft again to make sure that there is only one log file. What you're looking at is most likely the .log.gz file, which is not encrypted, but compressed.

the fml-client-latest  is very long how much of it do u need.

We need the end of the file. The 200 last lines would suffice.
I have tested in my side, I didn't get any error from Schematica.

It is a strange thing a lot of people are able to use it but their are still a few who can't including me and an admin on the server. http://pastebin.com/jbDbpmdw

ok

http://pastebin.com/Stws0Lep

It seems you are on the server when this occurs, please try in singleplayer.

that was in singleplayer the server is down

Yeah, I'm an idiot, I have read the log so fast.
I don't know what to do except try with the latest version of java.

I found something which would help @Lunatrius (or me) to find the reason for this bug, http://pastebin.com/f2Mt4ZS0 , start in line 83:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J 10472  org.lwjgl.opengl.GL11.nglDrawArrays(IIIJ)V (0 bytes) @ 0x0000000003c37bce [0x0000000003c37b80+0x4e] j  org.lwjgl.opengl.GL11.glDrawArrays(III)V+20 j  com.github.lunatrius.schematica.client.renderer.RendererSchematicChunk.updateRenderer()V+306 j  com.github.lunatrius.schematica.client.renderer.RendererSchematicGlobal.sortAndUpdate(Lcom/github/lunatrius/schematica/client/world/SchematicWorld;)V+59 j  com.github.lunatrius.schematica.client.renderer.RendererSchematicGlobal.render(Lcom/github/lunatrius/schematica/client/world/SchematicWorld;)V+124 j  com.github.lunatrius.schematica.client.renderer.RendererSchematicGlobal.onRender(Lnet/minecraftforge/client/event/RenderWorldLastEvent;)V+52 j  cpw.mods.fml.common.eventhandler.ASMEventHandler_1150_RendererSchematicGlobal_onRender_RenderWorldLastEvent.invoke(Lcpw/mods/fml/common/eventhandler/Event;)V+11

I updated my java but that didn't resolve the issue.

LWJGL is the cause, but i forget something, you are on 1.7.10 version. I don't think Lunatrius want to update Schematica for this minecraft version.
I have searched for a few, the only thing i can tell you now is to update your graphic driver: https://downloadcenter.intel.com/ .

It's a native crash inside nglDrawArrays which makes debugging and trying to reproduce it a difficult task. I can't remember if anyone ever found a solution to this, but I'd try updating the graphics drivers and possibly use MultiMC to override/update LWJGL.

I actually found something. Searching for ig75icd64 (filename part of C  [ig75icd64.dll+0x11e8a]) got me this Minecraft issue as the first hit.

ok ill see if it works

@snoozeville's crash log from #151: http://pastebin.com/PSXYM2vQ
What I would strongly suggest is to try running the game through your dedicated GPU if you have one (NVIDIA/AMD) and not through the integrated Intel GPU. I would also strongly suggest to read this issue from the Minecraft bug tracker.

How do u tell it which one to use

How would you accomplish that though? I've tried looking online and the methods some people have said don't work

This video shows how to do it. However, for Minecraft you have to add the appropriate java.exe or javaw.exe instead of the launcher.
In my case I have to point it to C:\Program Files\Java\jre1.8.0_74\bin\java.exe. The path to your java.exe file will differ depending on the version of Java (1.8.74 in my case) and whether it's 32 or 64 bit Java (Program Files (86x) vs Program Files).

i don't have nvidia or amd is there another work around

This is the only other solution, other than upgrading/downgrading drivers: https://bugs.mojang.com/browse/MC-32606

For 1.9 Users
If you are experiencing this issue after upgrading to 1.9, but did not for older versions of the game, the following instructions may solve the issue:

Increase allocated RAM from 1GB to 2GB (only applicable to 64 bit users)

In the Edit Profile menu in the Launcher, check the JVM Arguments box, and change the -Xmx1G argument to -Xmx2G

Change Video Settings in-game (Options -> Video Settings)

Enable VSync
Enable VBOs

If all of these did not correct the issue, follow the remainder of the instructions on this page, or seek assistance from the Mojang Support Center.


Memory Grid not refreshed on apply.

Hi
I hide the memory grid for new dialog. On saving of the dialog, I inserted some row in table related to memory grid by executing stored procedure.
But when apply button is clicked for save on new dialog, form get refreshed and memory grid is displayed but rows are not comes in memory grid. After closing the dialog and reopen it work fine.
Here is screen for more help -

Please help to resolve it. I am using Serenity <IP_ADDRESS>.

After save completes, dialog latest entity by calling Retrieve service, see why your retrieve service returns empty values.

Thanks @volkanceylan for reply!
I check the Retrieve service and that is return the item rows for grid correctly and these row are also available in loadEntity event of InvoiceDialog.ts but not shown in the grid.
Here is the screen shot of debug value of ItemList in loadEntity event -

How do you hide create that grid. You are probably doing something non-standard way.

Hmmm, You seems right!
I am hiding the grid with its category in following way -
this.form.ItemList.element.closest(".category").toggle(!this.isNewOrDeleted());

If i comment above line then grid shows at the time of creation and row is also inserted successfully in the grid after save. What is the correct way to hide the memory grid with its category name?

Okk.. Hiding the grid category using toggle is causing the issue.
So temporary i done following workaround which is working. Instead of using toggle i am adding and removing the class -
 if (this.isNewOrDeleted()) {
                this.form.ItemList.element.closest(".category").addClass("hideGrid"); //.toggle(!this.isNewOrDeleted());
            }
            else {
                this.form.ItemList.element.closest(".category").removeClass("hideGrid");
            }

And here is the css class where i set position of the category out from the dialog :-) :-)
.hideGrid {
    position:absolute;
    top:-200000px;
    width:100%;
}

I am sure this is not the correct way and there should be another standard way to do it.
Please suggest the correct way for this issue..

Try using .css('visibility', 'hidden'). I guess grid has data but it can't show rows as slickgrid requires resizeCanvas call when it becomes visible to calculate available space. It probably still thinks it has 0 pixel height when you toggle it.
Another option is to this.form.ItemList.element.closest(".category").toggle(!this.isNewOrDeleted()).trigger('layout') so SlickGrid can resize its canvas

Uhm no
this.form.ItemList.element.closest(".category").toggle(!this.isNewOrDeleted());
this.form.ItemList.element.trigger('layout');

Thanks @volkanceylan
Following is working perfectly -
this.form.ItemList.element.closest(".category").toggle(!this.isNewOrDeleted());
this.form.ItemList.element.trigger('layout');
Although .css('visibility', 'hidden') is also working. I already tried it instead of toggle but at that time this was not working, Probably i might doing something wrong or may be caching issue.

@ramveersgh
It's too awesome , How to achieve the Print button

@thisXYH
Do you want to add a new button on dialog or print functionality?

want to add a new print button on dialog .  in Seren Template , I only know a Sample

Not seen implemented in the dialog，

Here is the sample to generate the invoice of order.
You can treat Invoice button as Print?? :-) You will be required wkhtmltopdf.exe in App_Data\Reporting.

0 - 0
wkhtmltopdf.exe  ,  my computer no this app   hmmm.
I thought you were just shot me give you ended up function buttons

I use the Google Translation to talk with you ,
Sometimes difficult to understand the semantics ，


WIP: Retry DNS failure.

Fixes #5301

@timabbott, I have added the retry logic.

I don't understand why one would have DNS issues resolving https://push.zulipchat.com.

Is this endpoint behind Cloudflare?

@gnprice, @showell, can you take a look?

Merged, after fixing the above comment.  Thanks @umairwaheed!


add seconds option to interval_units with tests

Fixes #28

Adds the 'seconds' option to RepeatableJob.UNITS
Adds test for seconds unit
Adds ignore rules for Pipfile so I didn't clutter the repo

Thanks for the review @tom-price, that's a good point about confusion. If a user was to schedule a task every x seconds they would need to run rq-scheduler like so: rqscheduler -i x to adjust for the default per minute checking of tasks. Would a note in the README be of use?

I'm still torn on this. So adding seconds with a
I don't think your suggestion of how to update the readme fits. The get_scheduler function defined in Django-RQ shown below allows for the interval to be passed in when called by Django-RQ-Scheduler, but so currently there's no way to pass in that interval value. It's not like it's a parameter you can add to RQ_QUEUES.
def get_scheduler(name='default', interval=60):
        """
        Returns an RQ Scheduler instance using parameters defined in
        ``RQ_QUEUES``
        """
        return Scheduler(name, interval=interval,
                         connection=get_connection(name))

I also don't think that failing silently is an option if the interval set on the job is lower than the scheduler's interval, or maybe isn't even a multiple of the interval frequency (unsure about that part).
In RepeatableJob functions like these below could be added to throw an error if a save is attempted that violates these conditions; however, the example below relies on a protected member of  RQ_Scheduler's Scheduler class and self.scheduler would need to be modified to pass in an interval if one other than the default was set, which means you could pull from there if wanting one below 60. I'm going to keep thinking about this.
    def clean(self):
        super(RepeatableJob, self).clean()
        self.clean_interval()

    def clean_interval(self):
        if self.scheduler()._interval > self.interval_seconds():
            raise ValidationError({
                'interval_unit': ValidationError(
                    _("job interval is set lower than queue's interval"), code='invalid')
            })

I think the code looks good.
Would like to see some documentation with a disclaimer.

@g3rd Thanks for checking this out, do you think the rq-scheduler interval issue needs to be fixed by this PR or just a disclaimer before this is merged?

@g3rd The rq-scheduler interval issue has been resolved on this PR by @tom-price
If everything looks good could you please merge.

@tom-price I have merged in your PR that resolves the duplicate migration. At @divipayhq we have been running this branch in production for nearly 12 months and have not seen any issues with the interval being in the realm of a few seconds. I'd love to see this released as the next version, please let me know if anything is needed.


Feat: Add ignored statuses

We previously could only set requiredStatuses. This is great if all ci checks run
on every PR. We want to conditionally run different workflows on different PRs, so
we can't make certain statuses requried.
Instead, let's create a new ignoredStatuses that the user can set so that pulldasher
ignores those even if they are failing.
QA:
We should verify that ignored statuses don't affect the ci blocked PRs. You can modify
/mnt/ebs/etc/pulldasher-dev.config.js and update pulldasher dev to make sure the
correct pulls are in (or out) of the CI Blocked column on pulldasher-dev.cominor.com.
Connects: https://github.com/iFixit/ifixit/issues/52034
cc @danielbeardsley

So according to this configuration,

should we not show the pass fail status of ifixit-nextjs-dev?

QA 🎬
PRs that are failing with checks in ignoredStatuses are no longer CI blocked.


jdupes: 1.10.4 -> 1.11

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

Success on aarch64-linux (full log)
Attempted: jdupes
Partial log (click to expand)
install  -m 0644                jdupes.1 /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11/share/man/man1/jdupes.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11
shrinking /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11/bin/jdupes
gzipping man pages under /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11/share/man/
strip is /nix/store/p9akxn2sfy4wkhqdqa3li97pc6jaz3r1-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11/bin
patching script interpreter paths in /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11
checking for references to /build in /nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11...
/nix/store/2f05485v8m1b8an3vdhpcvkxzil1v3p3-jdupes-1.11

Success on x86_64-linux (full log)
Attempted: jdupes
Partial log (click to expand)
install  -m 0644                jdupes.1 /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11/share/man/man1/jdupes.1
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11
shrinking /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11/bin/jdupes
gzipping man pages under /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11/share/man/
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11/bin
patching script interpreter paths in /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11
checking for references to /build in /nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11...
/nix/store/hjad3dl2hnirv5kicd02pzqa10bxw211-jdupes-1.11

Success on x86_64-darwin (full log)
Attempted: jdupes
Partial log (click to expand)
test -e /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/bin || mkdir -p /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/bin
test -e /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/share/man/man1 || mkdir -p /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/share/man/man1
install  -m 0755        jdupes   /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/bin/jdupes
install  -m 0644                jdupes.1 /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/share/man/man1/jdupes.1
post-installation fixup
gzipping man pages under /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/share/man/
strip is /nix/store/g5r4apl0za012ffs6ladinwa5w0m1l3k-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11/bin
patching script interpreter paths in /nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11
/nix/store/ikv522h2gm3sxlhiyi56zrwckk892avs-jdupes-1.11