Move custom-package-server to separate repo

Quick Summary:
Cabal project kept giving me weird issues with building this project. The API for the package server also has stabilized and it's actually pretty useful as a moderating force to make sure that we are conservative with changes to the package server API that we keep it in a separate repository.
Having it in a separate repository also makes versioning and CI stuff a little bit more manageable.
SSCCE
N/A
Additional Details
If there was a way of getting nested cabal.project files to work, or at least to have nested cabal projects to work I'd be more inclined to keep it in this repo. But as it is, it's also just easier to have an independent release cycle for the package server separate from the compiler.

New repository is at https://github.com/Zokka-Dev/zokka-custom-package-server (no README yet but I'll fix that).


Where are the speed numbers for shift drag vs normal drag?

Where in the code does this get decided? Having looked at the code for hours, still can't find it.

imgui_widgets.cpp look for DragBehaviorT function and adjust_delta variable under g.IO.KeyAlt or KeyShift if statements

Yeah, it's decided by the DragBehaviour. You pass in a speed and shift makes it x times faster (can't remember the factor). You might be able to set an ImGui value somewhere to adjust the shift speed up factor.


Disable spammy gRPC logs

Currently, on a connection failure to CA, we log messages like
2020-06-08T23:52:39.466564Z     info    Subchannel Connectivity change to CONNECTING
2020-06-08T23:52:39.466642Z     info    Subchannel picks a new address "localhost:15012" to connect
2020-06-08T23:52:39.466789Z     info    pickfirstBalancer: HandleSubConnStateChange: 0xc000a3a2f0, {CONNECTING <nil>}
2020-06-08T23:52:39.466860Z     info    Channel Connectivity change to CONNECTING
2020-06-08T23:52:39.468138Z     info    grpc: addrConn.createTransport failed to connect to {localhost:15012  <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused". Reconnecting...
2020-06-08T23:52:39.468194Z     info    Subchannel Connectivity change to TRANSIENT_FAILURE
2020-06-08T23:52:39.468321Z     info    pickfirstBalancer: HandleSubConnStateChange: 0xc000a3a2f0, {TRANSIENT_FAILURE connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused"}
2020-06-08T23:52:39.468359Z     info    Channel Connectivity change to TRANSIENT_FAILURE
2020-06-08T23:52:39.591539Z     warn    cache   resource:default request:4f4ac8af-b0a6-4c4d-a5b6-fb79881fa896 CSR failed with error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused", retry in 400 millisec
2020-06-08T23:52:39.591741Z     error   citadelclient   Failed to create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused"

A few problems: there are 10 log lines for a single issue, and most of them have absolutely no context, nor are they good user faciung log messages. We already handle all gRPC level errors with our own contextual and user friendly(ish) errors, so we should just stick to those.

Instead, we can set log.Options.LogGrpc = false

That flag doesn't actually work to disable grpc logging, it just doesn't make it use the istio logging package

@howardjohn Can you elaborate? I noticed it set grpc logger here:
	// capture gRPC logging
	if options.LogGrpc {
		grpclog.SetLogger(zapgrpc.NewLogger(captureLogger.WithOptions(zap.AddCallerSkip(2))))
	}

if it's false it doesn't disable grpc logs though - they are enabled by default

From here
func init() {
	SetLoggerV2(newLoggerV2())
}

// newLoggerV2 creates a loggerV2 to be used as default logger.
// All logs are written to stderr.
func newLoggerV2() LoggerV2 {
	errorW := ioutil.Discard
	warningW := ioutil.Discard
	infoW := ioutil.Discard

	logLevel := os.Getenv("GRPC_GO_LOG_SEVERITY_LEVEL")
	switch logLevel {
	case "", "ERROR", "error": // If env is unset, set level to ERROR.
		errorW = os.Stderr
	case "WARNING", "warning":
		warningW = os.Stderr
	case "INFO", "info":
		infoW = os.Stderr
	}

	var v int
	vLevel := os.Getenv("GRPC_GO_LOG_VERBOSITY_LEVEL")
	if vl, err := strconv.Atoi(vLevel); err == nil {
		v = vl
	}
	return NewLoggerV2WithVerbosity(infoW, warningW, errorW, v)
}

google.golang.org/grpc/grpclog/grpclog.go

hmm that is weird I tried it but it did not work

@istio/wg-networking-maintainers

@howardjohn so what will be logged now instead of 10 lines of not helpful messages?

@nrjpoddar the last two lines are the Istio lines, the first 8 are grpc. So it would look like
2020-06-08T23:52:39.591539Z     warn    cache   resource:default request:4f4ac8af-b0a6-4c4d-a5b6-fb79881fa896 CSR failed with error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused", retry in 400 millisec
2020-06-08T23:52:39.591741Z     error   citadelclient   Failed to create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp [::1]:15012: connect: connection refused"

Note - these log lines are also not great, but thats another issue


Quit command shorthand /q

Added shorter version of quit command /q

This can already be done with the alias plugin:
/alias q /quit


Static build

Duplicate of #220 in attempt to get Netlify to work.

LGTM - can we merge this, @robmoffat , @TheJuanAndOnly99 , @maoo ?

I made her do it!


Help me with beforeDeploy

I'm trying to compile *.pug templates to *.html before deploy.
I have a script which I run like node build.js and it works good. But when I'm trying to use it in section beforeDeploy nothing happens.
here is my working script:
            var fs = require('fs');
            var pug = require('pug');
            var glob = require('glob');
            var path = require('path');

            console.log('building *.pug');

            glob('*.pug', {}, function(error, files){
                files.forEach(function(file, index){
                    var htmlFilename = path.basename(file, '.pug')+'.html';
                    console.log(file+' => '+htmlFilename);
                    fs.writeFile(htmlFilename, pug.renderFile(file, {pretty: true}));
                });
            });

and here is my beforeDeploy part:
                "beforeDeploy": [
                    {
                        "type": "script",
                        "script": "./build.js"
                    }
                ]

and here is my build.js script for vs-deploy:
console.log('test');

function execute(args) {
    console.log('execute starts');
    return new Promise(function(resolve, reject) {
        try {
            // do the MAGIC here

            // from the configuration example
            // above 'args.options' would contain
            // 5979 as value

            var fs = require('fs');
            var pug = require('pug');
            var glob = require('glob');
            var path = require('path');

            console.log('building *.pug');

            glob('*.pug', {}, function(error, files){
                files.forEach(function(file, index){
                    var htmlFilename = path.basename(file, '.pug')+'.html';
                    console.log(file+' => '+htmlFilename);
                    fs.writeFile(htmlFilename, pug.renderFile(file, {pretty: true}));
                });
                resolve();  // tell that anything went fine
            });

        }
        catch (e) {
            reject(e); // Oops, an error!
        }
    });
}
exports.execute = execute;

So I have no console outputs, no compilation of my *.pug and no errors:
[BEFORE DEPLOY #1]
Deploying file ..... [OK]
Finished

So what I'm doing wrong?

@LexRiver
Try to wrap the code with a try-catch-block and see if an error messages is shown as popup:
var vscode = require('vscode');

try {
    var fs = require('fs');
    var pug = require('pug');
    var glob = require('glob');
    var path = require('path');

    // console.log('building *.pug');
    vscode.window.showInformationMessage('build.js: building *.pug');

    glob('*.pug', {}, function(error, files){
        try {
            files.forEach(function(file, index) {
                var htmlFilename = path.basename(file, '.pug')+'.html';
                console.log(file+' => '+htmlFilename);
                fs.writeFile(htmlFilename, pug.renderFile(file, {pretty: true}));
            }
        });
        catch (e) {
            vscode.window.showErrorMessage('build.js.forEach(): ' + e);

            reject(e);
        }
        resolve();  // tell that anything went fine
    });
}
catch (e) {
    vscode.window.showErrorMessage('build.js: ' + e);

    reject(e);
}

I think the problem is that you have to link the pug module inside your project folder. It is not part of the extension.
For console.log you have to look at the debug console (CTRL + SHIFT + Y).

@LexRiver
I has the problem when I removed the pug part, the script told me that glob could not be loaded.
The problem is that this is no NodeJS API module.
After I replaced
var glob = require('glob');

with
var glob = args.require('glob');

it works.
The require method of args loads modules from the script / extension context, what means that you are also able to load modules that are part of the extension (and not only the VSCode / NodeJS ones).
These are the modules, the extension currently uses (s. package.json)
{
"dependencies": {
    "aws-sdk": "^2.7.19",
    "azure-storage": "^1.4.0",
    "fs-extra": "^1.0.0",
    "ftp": "^0.3.10",
    "glob": "^7.1.1",
    "html-entities": "^1.2.0",
    "i18next": "^4.1.4",
    "isbinaryfile": "^3.0.2",
    "less": "^2.7.2",
    "mime": "^1.3.4",
    "moment": "^2.17.1",
    "mssql": "^3.3.0",
    "mysql": "^2.12.0",
    "node-uuid": "^1.4.7",
    "node-zip": "^1.1.1",
    "nodemailer": "^2.7.0",
    "ssh2-sftp-client": "^1.0.5",
    "typescript": "^2.1.5"
  }
}

@LexRiver
Now it seems to work.
Open your terminal / command line, go to the folder where your build.js is stored and execute
´bash
npm link pug

It will create a `node_modules` subfolder with the module there.

@mkloubert
Thank you, compilation of *.pug files works now, but what are the correct settings to deploy these files to server?
Seems like they are deploying only on Ctrl+Alt+W, but not on Ctr+Alt+F

@mkloubert
If you could add some option like

track any file change in directory and deploy these files
that will be really great.

@LexRiver
There is still a deploy on change feature :-)
But before we start: Upgrade at least to version 5.19.1
1. We have to compile the PUG files to HTML
Create a package that tracks the .pug files:
{
    "deploy": {
        // ...

        "packages": [
            // ...

            {
                "name": "PUG to HTML",
                "description": "The files that should be compiled to HTML files.",

                "files": [
                    "/**/*.pug"
                ],

                "deployOnSave": [ "PUG Compiler" ]
            }

            // ...
        ]

        // ...
    }
}

This will send all .pug files to the following target, called PUG Compiler, automatically if you save one (or more):
{
    "deploy": {
        // ...

        "targets": [
            // ...

            {
                "name": "PUG Compiler",
                "description": "Compiles an HTML file to PUG",

                "type": "script",

                "script": "./build.js"
            }

            // ...
        ]

        // ...
    }
}

This script target will use your ./build.js as target, but first you have to modify it, because it has another structure:

function deployFile(args) {
    return new Promise(function(resolve, reject) {
        var onCompleted = function(err, hasCancelled) {
            args.canceled = hasCancelled;  // tell the extension
                                           // that we have cancelled

            if (err) {
                reject(err);
            }
            else {
                resolve(args);  // send input arguments
                                // back to the extension
            }
        };

        if (args.context.isCancelling()) {
            onCompleted(null, true);  // cancellation request
            return;
        }

        try {
            var fs = require('fs');
            var path = require('path');
            var pug = require('pug');

            var file = args.file;

            var htmlFilename = path.basename(file, '.pug') + '.html';
            var htmlFile = path.join(path.dirname(file),
                                     htmlFilename);  // I hope that HTML files
                                                     // should be created in
                                                     // same directory as PUGs

            if (args.deployOptions.onBeforeDeploy) {
                // use this to tell the extension that
                // we begin to deploy the file

                args.deployOptions.onBeforeDeploy(args.sender, {
                    destination: htmlFilename,  // this is only a string
                                                // to display in the output window
                    file: file,
                    target: args.target
                });
            }

            // render frpm PUG to HTML
            var pugData = pug.renderFile(htmlFilename, {
                pretty: true
            });

            // write to HTML file async
            fs.writeFile(htmlFile, pugData, (err) => {
                onCompleted(err);  // tell extenstion that we have finished
            });
        }
        catch (e) {
            onCompleted(e);
        }
    });
}

exports.deployFile = deployFile;

Now, when you save PUG files, it automatically compiles then to HTML.
2. Automatically deploy HTML after compiled
When a HTML file changes, you can use the deploy on change feature:
Add a second package that tracks changed HTML files:
{
    "deploy": {
        // ...

        "packages": [
            // ...

            {
                "name": "HTML from PUG",
                "description": "The files that should be deployed after compilation.",

                "files": [
                    "/**/*.html"
                ],

                "deployOnChange": true,

                "deployOnSave": [ "HTML target" ]
            }

            // ...
        ]

        // ...
    }
}

This will send all modifed / created HTML to the second target, called HTML target, we also have to create:
{
    "deploy": {
        // ...

        "targets": [
            // ...

            {
                "name": "HTML target",
                "description": "Deploys HTML files",

                "type": "test"
            }

            // ...
        ]

        // ...
    }
}

In my example I only used a mock target. Have a look at the wiki where all target types are listed.

@mkloubert
Thank you very much, Marcel.
That's works for me.
But small mistake in your code
           // render from PUG to HTML
            var pugData = pug.renderFile(htmlFilename, {
                pretty: true
            });

htmlFilename`` should be file```
and that's is not obvious that it's possible to use targets in deployOnSave, I think you should mention this in wiki.
"deployOnSave": [ "HTML target" ]

and another proposal is to make this combination of two targets to work a little faster.
When I save a *.pug file it compiles it to *.html and then waits for 1 or 2 seconds and only then deploying it.
So maybe this delay could be reduced.

@LexRiver
Thats right, it needs a better documentation.
I described it here, but the better way is to write it down to a separate wiki article, were you find examples for such scenarios. I will do this the next days.
With the speed problem, you seem to have the same problem as @rshakespear
I realeased version 5.20.0 that should reduce the lag, as you can read here.

@mkloubert
Another strange thing I've noticed.
If I compile pug templates outside of vscode when this deployment is on I'm getting strange warning messages:
Could not deploy '/file.html' to $'sftp' on save: Error: read ECONNRESET
or
Could not deploy '/file.html' to $'sftp' on save: Error: Timed out while waiting for handshake

Hi, before deploy a want call "gulp build" command for workspace, how i must write it in section  "beforeDeploy": []

@sag3ll0
You can use the open operation type:
{
    "deploy": {
        // ...

        "targets": [
            {
                "name": "My target",
                "type": "test",

                "beforeDeploy": [
                    {
                        "type": "open",
                        "target": "gulp", "arguments": [ "build" ]
                    }
                ]
            }
        ]
    }
}

If the extension cannot execute your gulp, try to set the full path or create a npm link inside your workspace:
npm link gulp

it working, but it call new cmd window, that after operation not close. Can we use vs code terminal?

@LexRiver I had same Error: Timed out while waiting for handshake
I found a solution by add a config variable "readyTimeout" in connect() function of ssh2-sftp-client
in out\src\plugins\sftp.js, line 179, add readyTimeout: 99999
                    conn.connect({
                        host: host,
                        port: port,
                        username: user,
                        password: pwd,
                        privateKey: privateKey,
                        passphrase: target.privateKeyPassphrase,
                        hostHash: hashAlgo,
                        hostVerifier: (hashedKey, cb) => {
                            hashedKey = toHashSafe(hashedKey);
                            if (hashes.length < 1) {
                                return true;
                            }
                            return hashes.indexOf(hashedKey) > -1;
                        },
                        agent: agent,
                        agentForward: agentForward,
                        tryKeyboard: tryKeyboard,
			readyTimeout: 99999 // to fix handshake timeout issue
                    })

@lvbeck
Great! I will add and release a new version that can define that value in about 2 or 3 hours.

@lvbeck @LexRiver
Since version 5.32.0 there is now a readTimeout property:
{
    "deploy": {
        // ...

        "targets": [
            {
                "name": "My target",
                "type": "sftp",

                "readyTimeout": 99999
            }
        ]
    }
}

@sag3ll0
Sorry, when I answer so late ... since version 7.0.0 I have implemented the feature of running apps and "bash" scripts inside the integrated terminal (by setting runInTerminal property to (true) in the operation):
{
    "deploy": {
        // ...

        "targets": [
            {
                "name": "My target",
                "type": "test",

                "beforeDeploy": [
                    {
                        "type": "open",
                        "target": "gulp", "arguments": [ "build" ],

                        "runInTerminal": true
                    }
                ]
            }
        ]
    }
}


About the SNLI datasets

Hi Tianyu,
Thanks for your code.  I found that the downloaded SNLI dataset is slightly different from the original one. The statistics are shown below:
yours:
9843 dev.tsv
9825 test.tsv
549368 train.tsv
569036 total
original:
10001 snli_1.0_dev.txt
10001 snli_1.0_test.txt
550153 snli_1.0_train.txt
570246 total
May I ask what changes have you made?
Regards,
Yiming

Hi Yiming,
It's because SNLI contains some instances with no label (the field of "gold label" is "-" because the annotators did not reach an agreement). We just filtered them out. Sorry that we didn't mention this in the paper & README, and we will update with the details later.


Update end-to-end-tests to use PR number instead of branch name

This PR changes the end-to-end tests, to use the PR number instead of the name of the branch as the suffix for the created repo. This has several advantages:

The name of the repo is not longer than the allowed maximum length.
The name of the repo clearly indicates which PR it refers to.
There is no chaos with branches of the same name from different repos.

Do we have to change anything for push or anything else @ewjoachim?

Concurrency might also be good to ensure that two end-to-end tests never run on the same branch at the same time. Something like this:
concurrency:
  group: ${{ github.head_ref || github.ref_name }}


Plugin isn't stable

This seems like the only spellcheck plugin in existence for sketch but it doesn't really work anymore in my experience. Intentional misspelled words are missed. No indication that the spellcheck ran when I initiate it. Just a warning to someone who might be relying on this plugin. I've been burned a few times by it.
I see @moderntarra is having the same problem. Anyone else having this issue?

sometime It will go beyond the particular page, spell check of override doesn't work all the time

I haven't been able to replicate the override not working issue, though yes, it probably checks other pages that you're not on.


Labels don't disappear on mobile platforms.

There is no hover in mobile Safari. Labels appear when a marker is clicked.
They disappear if you tap on a map, however if you directly tap on another marker a new label opens and the existing one doesn't close.
Is there any way of closing existing labels when a new one opens?

Did you find any solution to this ?
I have been experiencing the same problem with touch devices.The label does not destroy when a new polygon is touched.

No, as I understand it, each marker and therefore label doesn't have a reference to all the other markers on your map.
For desktop this doesn't matter because there is a mouseout event, which causes the current label to close. I don't believe there is an equivalent for touch.
In your app, if you maintain an array of all your markers you should be able to call the .hideLabel method on them before you open a new one. It would mean hacking
src/BaseMarkerMethods.js
to contain something like :
_addLabelRevealHandlers: function () {
		this
			.on('mouseover', this.showLabel, this)
			.on('mouseout', this.hideLabel, this);

		if (L.Browser.touch) {

			// loop through all the markers and close their labels.
			yourMarkerArray.forEach(function(marker) {
			   marker.hideLabel;
			});

			this.on('click', this.showLabel, this);
		}
	}

That said I've not tested it, so good luck.
I'd probably settle for being able to turn off labels for touch.

+1 for this issue, I am going to have to create a similar workaround for our mobile users. It would be great if Leaflet.label had some kind of built-in functionality for touchscreen interactions.

In lieu of a fix to this library, one possible solution is to use L.Browser.touch to test for touch device support, and then bind a Leaflet popup for touch browsers and labels for non-touch browsers.


[Docs] Add InstructLab LLM guide to docs

Description: Adds a model support list to the InstructLab documentation
Preview: https://github.com/kelbrown20/kelly-instructlab/blob/create-models-chart/docs/instructlab_models.md

Requesting review but putting on hold because this PR needs to be updated after #2585 merges


loot command doesn't accept loot tables

Why don't you use the builtin method myLootTable.give('@a')?
I will still fix it, but using the builtin method is better here.

Closer to vanilla syntax => more readable to people outside the project


Add an issue template

Since we will be adding our own issues as well now, it will be better and more convenient if we have an issue template that we can follow to provide specific, structured information when we open issues.
If you feel this is something we can add to the repo, please feel free to assign it to @SciencePotato and me.

Not a major use case


what is type of licence

what is type of licence

I just put it under MIT license. Should be compatible to Apache-2.0 license from pvporcupine and the modified BSD of torch. All other dependencies are MIT too.


undefined method `add_foreign_key'

rake aborted!
NoMethodError: undefined method `add_foreign_key' for #<ActiveRecord::Schema:0x007f9688f7f9b0>
/Users/justin/workspace/scholarsphere/db/schema.rb:304:in `block in <top (required)>'
/Users/justin/workspace/scholarsphere/db/schema.rb:14:in `<top (required)>'
Tasks: TOP => db:schema:load

If I do "rake db:migrate" it makes a different db/schema.rb with this at the end:
-  add_foreign_key "mailboxer_conversation_opt_outs", "mailboxer_conversations", name: "mb_opt_outs_on_conversations_id", column: "conversation_id"
-
-  add_foreign_key "mailboxer_notifications", "mailboxer_conversations", name: "notifications_on_conversation_id_development", column: "conversation_id"
-
-  add_foreign_key "mailboxer_receipts", "mailboxer_notifications", name: "mailboxer_receipts_on_notification_id_development", column: "notification_id"
-
+  Foreigner.load

Should I commit that?

This may be because I'm running sqlite and you are running mysql maybe?

@awead do you have any details on this resolution? Is this just stale?

@jcoyne yeah, it's like the schema.rb was committed at some point, otherwise, we haven't seen this error.


updated pictures

👷 Deploy Preview for semanticclimate processing.

Name
Link

🔨 Latest commit
e05127b963c627afe9527d272303f391937112fe

🔍 Latest deploy log
https://app.netlify.com/sites/semanticclimate/deploys/64ba53bf1b770b00080b7ce1

update photo

new photo added


The input requirements for a custom Form Recognizer model are the same as the general Form Recognizer input requirements

The input requirements for a custom model are here
), and as follows:
Form Recognizer works on input documents that meet these requirements:

    Format must be JPG, PNG, PDF (text or scanned), or TIFF. Text-embedded PDFs are best because there's no possibility of error in character extraction and location.
    If your PDFs are password-locked, you must remove the lock before submitting them.
    PDF and TIFF documents must be 200 pages or less, and the total size of the training data set must be 500 pages or less.
    For images, dimensions must be between 600 x 100 pixels and 4200 x 4200 pixels.
    If scanned from paper documents, forms should be high-quality scans.
    Text must use the Latin alphabet (English characters).
    For unsupervised learning (without labeled data), data must contain keys and values.
    For unsupervised learning (without labeled data), keys must appear above or to the left of the values; they can't appear below or to the right.

Form Recognizer doesn't currently support these types of input data:

    Complex tables (nested tables, merged headers or cells, and so on).
    Checkboxes or radio buttons.

The general input requirements for Form Recognizer are here and as follows:
Form Recognizer works on input documents that meet these requirements:

    Format must be JPG, PNG, PDF (text or scanned), or TIFF. Text-embedded PDFs are best because there's no possibility of error in character extraction and location.
    If your PDFs are password-locked, you must remove the lock before submitting them.
    PDF and TIFF documents must be 200 pages or less, and the total size of the training data set must be 500 pages or less.
    For images, dimensions must be between 600 x 100 pixels and 4200 x 4200 pixels.
    If scanned from paper documents, forms should be high-quality scans.
    Text must use the Latin alphabet (English characters).
    For unsupervised learning (without labeled data), data must contain keys and values.
    For unsupervised learning (without labeled data), keys must appear above or to the left of the values; they can't appear below or to the right.

Form Recognizer doesn't currently support these types of input data:

    Complex tables (nested tables, merged headers or cells, and so on).
    Checkboxes or radio buttons.

There are exactly the same, is this a mistake? If it isn't then IMO simply having something like "At the present time, the input requirements for a custom Form Recognizer match the general input requirements, you can view them here.(link to general requirements section)" Rather than stating them again verbatim and user's having to look through them for differences or getting confused.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 400d903c-36d4-e20b-d2fc-b80b53b00ceb
Version Independent ID: 210dbaf5-eda5-f50e-8d8e-18b86daff035
Content: What is Form Recognizer? - Azure Cognitive Services
Content Source: articles/cognitive-services/form-recognizer/overview.md
Service: cognitive-services
Sub-service: forms-recognizer
GitHub Login: @PatrickFarley
Microsoft Alias: pafarley

@BeigeBadger Thanks for the inputs. Yes, these are currently the same requirements in both the cases. We would request our content author @PatrickFarley to check if they can be merged. @NHaiby FYI

Thanks for the confirmation @RohitMungi-MSFT.

@PatrickFarley Can we address this as part of the next doc update to merge the requirements to remove any redundant information?

Good catch; I've reworded the doc so it's clear that those input reqs are really only for custom model training.
#please-close


<EMAIL_ADDRESS>fails

npm install is failing for me. It looks like there is an issue with node-gyp or the fact that I am using Node 0.12.0. Any recommendations to resolve? Here is the log
$ npm install
npm WARN package.json<EMAIL_ADDRESS>No repository field.
npm WARN package.json<EMAIL_ADDRESS>No repository field.
npm WARN package.json<EMAIL_ADDRESS>No repository field.
npm WARN engine<EMAIL_ADDRESS>wanted: {"node":"0.10.x"} (current: {"node":"0.12.0","npm":"2.5.1"})

<EMAIL_ADDRESS>install /Users/tyler.harrington/Desktop/express4-bootstrap-starter/node_modules/email-templates/node_modules/juice2/node_modules/jsdom/node_modules/contextify
node-gyp rebuild

child_process: customFds option is deprecated, use stdio instead.
CXX(target) Release/obj.target/contextify/src/contextify.o
../src/contextify.cc:34:17: error: no member named 'ContextDisposedNotification'
in 'v8::V8'
v8::V8::ContextDisposedNotification();
~~~~~~~~^
1 error generated.
make: *** [Release/obj.target/contextify/src/contextify.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack     at ChildProcess.emit (events.js:110:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1067:12)
gyp ERR! System Darwin 14.1.0
gyp ERR! command "node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/tyler.harrington/Desktop/express4-bootstrap-starter/node_modules/email-templates/node_modules/juice2/node_modules/jsdom/node_modules/contextify
gyp ERR! node -v v0.12.0
gyp ERR! node-gyp -v v1.0.2
gyp ERR! not ok
npm ERR! Darwin 14.1.0
npm ERR! argv "node" "/usr/local/bin/npm" "install"
npm ERR! node v0.12.0
npm ERR! npm  v2.5.1
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node-gyp rebuild'.
npm ERR! This is most likely a problem with the contextify package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls contextify
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/tyler.harrington/Desktop/express4-bootstrap-starter/npm-debug.log
tylerharrin-ltm:express4-bootstrap-starter tyler.harrington$

@tharrington i not testing it yet with node 0.12.0

it's not working with node > 0.12.0. Install nvm and make: nvm use 0.10.26

@tharrington @romabelka - I got it working on 0.12.0 by upgrading Mongoose to 4.1.3. I renamed npm-shrinkwrap.json for a quick test and it looks good. Im guessing the dependencies in shrinkwrap.json need to be updated.


Errors when running script

I've tried to install and uninstall a couple of times.
I'm on a Windows system with Git installed and I'm not a developer. I'm running the script in Git Bash.

As standard Make and Wget isn't installed with Git on Windows, so needs to be downloaded separately.

There is an error later in the script that seems to relate to replacing a temporary location with an actual location:
go mod edit -replace github.com/go-skynet/go-llama.cpp=/c/autogpt4all/LocalAI/go-llama
process_begin: CreateProcess(NULL, go mod edit -replace github.com/go-skynet/go-llama.cpp=/c/autogpt4all/LocalAI/go-llama, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [Makefile:175: replace] Error 2

The rest of the script seems to be running fine, but the end result is that there's no local-ai file, which I'm supposed to run and there is no downloaded model. I think it may be related to point 2.

@FrederikSchack ill keep this open for now, will probably replace the bash script with python so there is better windows compatibility, will keep you in loop on when its available

Thanks!

@Smartking1 @FrederikSchack
check latest readme, there are tweaks on script version and there is a python script alternative


Home page banner/landing section #310

issue #310
Done, as I showed during our team meeting.

Still need to fix eslint errors. Holding off on merging until I fix the eslint errors.

fixed eslint errors. Should be ready to merge now.

@esteban-gs Refactored as much as it currently make sense to me.
Should be ready to merge, but also let me know if you need anything else.


Styled save and cancel button

What is the change?
1.Added required classes to readprofile
2. added --color-danger:#ff0000
3. Added button-cancel class
Related issue?
close: #716
How was it tested?
Remove this line and tell us how the code was tested.
Checklist:
Before you create this PR, confirm all the requirements listed below by checking the checkboxes [x]:

[x] Have you followed the Contribution Guidelines while contributing.
[x] Have you checked there aren't other open Pull Requests for the same update/change?
[] Did you lint your code before making the Pull Request?
[] Have you made corresponding changes to the documentation?
[x] Your submission doesn't break any existing feature.
[x] Have you tested the code before submission?

Screenshots or Video:
Add a screenshot or demo video if appropriate.

/gssoc/

/level2/


bug Minor fixes for wikisum (Windows 10 and Python 3)

Description
I can currently not sign the Contributor License Agreement, so I will not do a pull request, sorry about this.
If you are unable to process any of these patches without a pull-request, I will make one at the end of the week. But for the sake of getting these notes out as soon as possible, for now, this is all I can offer.
Notes on shell commands
(https://github.com/tensorflow/tensor2tensor/tree/master/tensor2tensor/data_generators/wikisum#commands-to-generate-wikisumweb)

Command "python -m tensor2tensor.data_generators.wikisum.parallel_launch" needs to go without '.py'

--command_prefix can't find the scripts, needs to do a cd first (will break --code_dir feature, but this seems to have no effect on the other commands anyway, as they directly call the python module)
--command_prefix="cd ~/.local/lib/python3.5/site-packages/tensor2tensor/data_generators/ ; python3 -m tensor2tensor.data_generators.wikisum.get_references_web --out_dir={{BUCKET}}/wiki_references --shard_id"

If "Cloud Storage JSON API" is not enabled yet in the gcloud account, it will fail silently on the cloud worker. Should be checked in parallel_launch.py first

Vocab-generation on Python3 throws an re exception

   File "/usr/local/lib/python3.5/dist-packages/tensor2tensor/data_generators/wikisum/wikisum.py", line 361, in _normalize_text
    text = re.sub("[%s]" % re.escape(string.punctuation), r" \g<0> ", text)
  File "/usr/lib/python3.5/re.py", line 182, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: cannot use a string pattern on a bytes-like object

Vocab-generation does not work on windows, as it can not access gs:// files, so it should be done in the cloud as well (explitictly using python2 to work around the re exception mentioned above)

python2 -m tensor2tensor.data_generators.wikisum.parallel_launch \
  --num_instances=1 \
  --cpu=4 --mem=16 \
  --name=wikisum-vocab-gen \
  --setup_command="pip install tensor2tensor tensorflow -U -q --user" \
  --command_prefix="python2 -m tensor2tensor.data_generators.wikisum.generate_vocab  --out_dir=$BUCKET/data  --refs_dir=$BUCKET/wiki_references ; echo Done shard

Changes for parallel_launch.py
Here is the full parallel_launch.py file https://pastebin.com/UUDjA9jL
Below are the changes I made:
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L46 - Cleaning output
#FIX prevent a ton of future warnings from h5py
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L88 - Some changes to the shell commands
#FIX added username 'wsumuser', otherwise got an error as my windows-username is root
COPY_CODE = "gcloud compute scp --recurse {local_dir} wsumuser@{instance_name}:~/"
SSH = "gcloud compute ssh wsumuser@{instance_name} --command"
DEFAULT_ZONE = "gcloud config get-value compute/zone"
#FIX use screens logging functionality to get rid of escaping problems for window's popen and to get more log coverage
SCREEN = "screen -L ~/logs-XXX.txt -dmS test bash -c \"{command}\""
#FIX no need for piping anymore, using screen -L
LOGS = "; gsutil cp ~/logs-XXX.txt {bucket}logs-{task_id}.txt"

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L105 - Find gcloud on windows
     #FIX try to find gcloud on windows
      try:
        return sp.check_call(args)
      except FileNotFoundError:
        if args[0] == "gcloud": args[0] = os.getenv('LOCALAPPDATA') + "/Google/Cloud SDK/google-cloud-sdk/bin/gcloud.cmd"
        return sp.check_call(args)

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L115 - Use python instead of netcat to wait for SSH
def wait_for_ssh(ip):
  """Wait for SSH to be available at given IP address."""
  # FIX don't use netcat, but python to test for open SSH port
  import socket
  for i in range(12):
    s = socket.socket()
    s.settimeout(2)

    try:
      s.connect((ip, 22))
      s.close()
      return True
    except:
      s.close()
    time.sleep(10)
  return False

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L209 - Python3 compatibility
  vm_names = list(zip(*vm_info))[0] if vm_info else [] #FIX: make a list from zip()-iterator first
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L216 - Log-Dir handling on windows
    #FIX window's \ leads to an error on the cloud-server afterwards
    log_dir = log_dir.replace("\\","/")

    #FIX on windows there is no gs:// , so give the user the opportunity to create the directories
    from tensorflow.python.framework.errors_impl import UnimplementedError
    try:
      tf.gfile.MakeDirs(log_dir)
    except UnimplementedError:
      input("[!] Use http://console.cloud.google.com and manually create '" + log_dir + "', then press return.................")

https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L251 - Don't fail silently (the zip-exception, for example, was silenced)

    # FIX Don't fail silently
    except Exception as e:  # pylint: disable=bare-except
      failed.append(i)
      tf.logging.error("Failed to launch task %d due to exception %s", i, str(e))

Changes for cloud_tpu.py
https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/utils/cloud_tpu.py#L229 - Find gcloud on windows
def shell_output(cmd_, **kwargs):
  try:
    return text_encoder.to_unicode(sp.check_output(format_cmd(cmd_, **kwargs)))
  except FileNotFoundError:
    return text_encoder.to_unicode(sp.check_output(format_cmd_win(cmd_, **kwargs)))

def shell_run(cmd_, **kwargs):
  try:
    return sp.check_call(format_cmd(cmd_, **kwargs))
  except FileNotFoundError:
    return sp.check_call(format_cmd_win(cmd_, **kwargs))

def format_cmd_win(cmd_, **kwargs):
  ret = cmd_.format(**kwargs).strip().split()
  if ret[0] == "gcloud": ret[0] = os.getenv('LOCALAPPDATA') + "/Google/Cloud SDK/google-cloud-sdk/bin/gcloud.cmd"
  elif ret[0] == "gsutil": ret[0] = os.getenv('LOCALAPPDATA') + "/Google/Cloud SDK/google-cloud-sdk/bin/gsutil.cmd"
  return ret

TensorFlow and tensor2tensor versions
tensor2tensor 1.6.1
tensorflow-gpu 1.7.0

Thanks very much @f-lng!
I'll address some of the bugs, though I'm not keen on adding Windows support currently. I'll have to think more about it since it adds a bit of complexity.
Were you able to generate the dataset in the end? Web or Commoncrawl or both?

Also: Reducing the timeout to 10 seconds saves 20% of computational costs, with neglectable decrease of coverage.
(Note: I only tested coverage on the first shard for both timeouts.)
(Here https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/get_references_web_single_group.py#L209 )

@rsepassi Yes, I figured that windows is not very high on the list, and it's no problem for me, but perhaps someone will find the notes usefull anyway. However, as most stuff happens in the cloud, it is not that much of a problem to adjust it accordingly. This is no "run and forget" generator anyway.

Not yet, I had to wait until yesterday to get my quota increase, so right now I am only as far as downloading all the references.
However, I did test the other parts on a small subset already, and I am positive it will work out in the end.
Tough, my version of wikisum is now a bit different from yours (due to changes because of windows, but also due to changes in how I want my dataset to be generated [using a bigram overlap per sentence, instead of tfidf per paragraph]).

Additional notes:
== Reliability of SSH commands ==
As soon as I try to spawn all 1k instances at once, even tough I reduced the threads to 10, I do get immediate and frequent SSH timeouts (they are retried, and mostly work out, but still).
What works better is to have a loop that calls parallel_launch.py with chunks of 10 using --instance_ids
== Deletion on error ==
This is related to the point above. I was lucky that I spawned in chunks of 10, as one time the retries did not work as well, and then the script tried to delete all previously spawned workers as well. I think this is not the best approach, as it might be common that at least one of them times out, and then you would have to start over.
== Checks ==
Also somehow related, I think there is the need for some additional checks after the download has been completed, in order to check if every shard has been downloaded correctly.
I will write one that parses the logs in the google bucket, but it will look quite messy (on windows I have to download all of the logs first, for example). Should I still post it here?
== Text normalization ==
Still not sure if it was a neccessary decision because of the capabilities of the transformer, but I think it should not include lowering the text, as - as far as I know - lowering the text on a 32k bilingual corpus is not necessary, so I think it is not necessary here as well. What do you think?
== code_dir upload ==
I think it should be uploaded directly into "~/.local/lib/python{python_version}/site-packages/tensor2tensor/data_generators/" , to make sure all imports use the new version and the "python -m" command work out in the same way, too

@rsepassi
For me it does not take 8 hours but only 16 minutes to generate the final traindata.
I tested on the first 10 shards, here is the log of the first 3 : https://pastebin.com/K3YxjuXk
The generated datafiles are ~200mb, so the 200gb for the final dataset seems about right.
They hold about 2.2k datapoints (ref->wiki) each, so in total I expect 2.2mio datapoints. Does that seem about right as well?

Deletion bug
If you start instances with --instance_ids , the deletion-on-failure routine mixes up the instance's index with its actual ID/Name, and tries to delete the wrong instances
Here you can see the problematic code https://pastebin.com/fuL2UZFw

Thank you so much for the further reports @f-lng. Keep them coming!
For the SSH reliability: I think the best thing to do is to try a full launch of the 1k and then do a relaunch with --instance_ids for the ones where the SSH command failed.
Checks: I tried to make the scripts that are launched on the remote machines as robust as possible so if some of your jobs failed after launch, please report the error messages here.
Text normalization: it may not be strictly necessary, but it's what we did for the paper so we kept it the same.
code_dir upload: If you use the code_dir feature, you'll want to update the setup command. For example, you can make code_dir the entire local tensor2tensor directory and change the setup command to pip install -e . --user so that it makes all the from tensor2tensor... imports use the local copy.
Timeout: that's great. I'll reduce it to 10 and update the cost.
Time and costs: Yes, it took <30m for the final step for me as well. I must have based the initial estimate on a shard running on my local machine instead of on GCP so the data transfer was the bottleneck. Will update the time and cost estimates provided.
Deletion bug: Good find! Will update.

@rsepassi Sure thing, but I think these were all the bugs I will find, as I am currently generating the final dataset. I will give you an update as soon as I have trained a model with it, tough.
SSH reliability: Indeed that might have been a better approach, it takes a lot of time to spawn them in chunks of 10.
Checks: I think the problems came from time-outed SSH commands, so nothing you can fix I guess. As a very simple check, perhaps this snippet is enough:
rerun_shards = set()
ret = shell_output("gsutil ls {bucket}", bucket=BUCKET + "/wiki_references/process_0")

for i in range(1000):
    if ret.find(".gz-"+str(i).rjust(5, '0')+"-of-01000") == -1:
        rerun_shards.add(i)

print("INSTANCES_REDO =" , list(rerun_shards) )

Text normalization: Alright, good to know, thanks!
code_dir upload: I fixed it by uploading the wikisum/-directory directly into the pip installation, so I did not have to upload the entire tensor2tensor-directory.
Time and costs: Good to know. So all in all, the costs were pretty ok, especially given the 250€ you get with a new google-cloud account.

Good luck on your experiments!
On Thu, May 3, 2018 at 12:07 PM Fabian Langer<EMAIL_ADDRESS>wrote:

@rsepassi https://github.com/rsepassi Sure thing, but I think these
were all the bugs I will find, as I am currently generating the final
dataset. I will give you an update as soon as I have trained a model with
it, tough.
SSH reliability: Indeed that might have been a better approach, it takes a
lot of time to spawn them in chunks of 10.
Checks: I think the problems came from time-outed SSH commands, so nothing
you can fix I guess. As a very simple check, perhaps this snippet is enough:
rerun_shards = set()
ret = shell_output("gsutil ls {bucket}", bucket=BUCKET + "/wiki_references/process_0")
for i in range(1000):
if ret.find(".gz-"+str(i).rjust(5, '0')+"-of-01000") == -1:
rerun_shards.add(i)
print("INSTANCES_REDO =" , list(rerun_shards) )
Text normalization: Alright, good to know, thanks!
code_dir upload: I fixed it by uploading the wikisum/-directory directly
into the pip installation, so I did not have to upload the entire
tensor2tensor-directory.
Time and costs: Good to know. So all in all, the costs were pretty ok,
especially given the 250€ you get with a new google-cloud account.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensor2tensor/issues/757#issuecomment-386404014,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABEGW560mOcKfxdbOSPL4NnodgjK-ebHks5tu1VbgaJpZM4TsUiW
.

For others that see this bug, the fixes will be out with the next T2T release sometime next week.

Yes, I’ll add a note on the length parameter. Hopefully we’ll have better
repro instructions on the next push.
On Fri, May 4, 2018 at 2:24 AM Fabian Langer<EMAIL_ADDRESS>wrote:

Just started the first training run @rsepassi
https://github.com/rsepassi and noticed the following things
eval_steps
--eval_steps=100
Seems a bit low, especially given the huge amount of evaluation data.
Looks like you would spend most time on evaluation with that setting.
hparams
There should be a note on setting the L parameter, as the paper mentions
that the base transformer is not able to learn for L ~ 2000 (and also
mentions the best range for L on lead section generation)
hparams.max_input_seq_length = 500
hparams.max_target_seq_length = 0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensor2tensor/issues/757#issuecomment-386548131,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABEGW0sm84fTkss6nBcYK6fScT2aQA4Pks5tvB4wgaJpZM4TsUiW
.

@rsepassi Not sure if I should start a new issue for this.. Anyway, here are my experiences for the first training run
Training
hparams and problem
I trained on WikisumWebLeadSection with these hparams:
@registry.register_hparams
def transformer_big_wsum_v1():
  hparams = transformer_big_single_gpu() # for best results
  hparams.batch_size = 2500 # maximum for 1080TI
  hparams.max_input_seq_length = 500  # setting L
  hparams.max_target_seq_length = 0 # never truncate EOS
  return hparams

Training data
To start simple, I implemented a "cheating extractor" that ranks each sentence in the reference by it's bigram overlap with the target, according to the paper. Afterwards I checked the generated files with a different implementation of a bigram-overlapp algorithm and they seem fine.
Loss and results
It seems to overfit pretty quickly (screenshot made after 20 hours and 6k steps) :
https://pasteboard.co/HjJxXt6.png
A qualitative analysis shows that it does not learn to use the input at all at this step, here is an example
https://pastebin.com/LLBXGGr2
A decode with the 4k-steps network is no better btw.
Next steps
I will try to train the smaller base version on your tfidf-dataset to get as close to your results as possible now.
Anything else you would advise?

@rsepassi I found a problem that will prevent training on wikisum data at all.
debug_keep_up
If the flag is set, the script checks for num_instances instead of the actual requested instance_ids to be 1. However, using --instance_ids you could set one specific instance to debug and still have num_instances at 1000, so len(instances_ids) should be checked.
So https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/data_generators/wikisum/parallel_launch.py#L248 should be
assert len(instance_ids) == 1
Failing generation due to out-of-memory
About 25 instances fail during example generation due to running out of memory.
I did not find the bug previously, as the according failure message only shows up in the log sometimes (mostly the process is killed silently).
But due to the next bug, I had to look  into it and found that, in order to make sure all datafiles are generated, you need 3gb of RAM instead of 2.
Silent failing if not all datafiles are there
What makes the above problem so severe is this:
Upon inspecting the output of t2t_trainer, I noticed that evaluation stopped at [60/100] each time.
I then noticed that "wikisum_web-test-00069-of-00100" was missing. And not only that, but also some training files.
However, there was no error thrown or anything, I only noticed because I saw evaluation stopping too early. It seems t2t simply ignores any other data file after the first missing one.
So...

Not all data files can be generated with default settings (while I did know about it, I figured it's ok if a few were missing as I had no time to look into it. Well, guess that was wrong)
tensor2tensor stops processing any more training files as soon as one is missing (at least in evaluation, but I guess in training it is the same)
The first file that was missing was 73 of 800 or so, and I guess it is the same for everyone trying to generate data with the default 2gb ram machines
The system will not learn and overfit quickly, as it only gets to see 9% of the actual data

Update:
Hmm that seems odd, evaluation still stops at 60..

INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [10/100]
INFO:tensorflow:Evaluation [20/100]
INFO:tensorflow:Evaluation [30/100]
INFO:tensorflow:Evaluation [40/100]
INFO:tensorflow:Evaluation [50/100]
INFO:tensorflow:Evaluation [60/100]
INFO:tensorflow:Finished evaluation at 2018-05-05-11:56:29

But this time I checked and have 1001 files in the data directory.

Training behaviour
@rsepassi
Alright, overfitting is no longer happening and the training behaviour looks good to me with:
@registry.register_hparams
def transformer_base_wsum_v1():
  hparams = transformer_base_v1()

  hparams.max_input_seq_length = 500
  hparams.max_target_seq_length = 0
  hparams.max_length = 500

  return hparams

However, I am not sure if the metrics are within expected ranges, could you check against your own training runs? https://pasteboard.co/Hk2M9A9.png
Collaps of model output
And while the model still seems to learn nicely I found something odd - and I am not sure if that is expected during early training (20k steps and 1.5 days in) or a sign for a problem: model output collapsed to one "best guess", and the model (4 beams, alpha 0.6) always outputs the same sentence, regardless of the input.
Here you find an example decode from the dataset:
https://pastebin.com/a540WiZ2

Thanks for the further notes @f-lng.
I'll fix the debug_keep_up bug and up produce_examples to 3G of memory.
I'll also add a data validation script and instructions for running it to the README.
On the issue of training stopping when a file is missing, I don't understand where that behavior is coming from and so am skeptical that that's indeed what you're seeing. To find files to read, it uses a filepattern, so it shouldn't matter if there are missing ones. Could you provide some more of your observations that leads you to believe that it's stopping because of a missing file?

@rsepassi No you are right, it was just an assumption that I made based on the fact that evaluation stopped at 60/100 and the first missing file was in that range as well.
But the assumption was wrong, it did not stop because a file was missing. It simply skipped the data due to "eval_drop_long_sequences=true"
Anyway, what is more important: Can you say anything about the training behaviour, my hyperparameters and the model collaps that I am seeing?
I am now 2.5 days in, and output seems to have collapsed to a single sentence.

@rsepassi No you are right, it was just an assumption that I made based on the fact that evaluation stopped at 60/100 and the first missing file was in that range as well.
But the assumption was wrong, it did not stop because a file was missing. It simply skipped the data due to "eval_drop_long_sequences=true"
--
@rsepassi @lukaszkaiser
What is more of an issue now is the actual training.
Can one of you say anything about the expected training behaviour, my hyperparameters and the model collaps that I am seeing? Is it expected after 2.5 days or is something broken?
I am now 2.5 days in, and output seems to have collapsed to a single sentence, no matter what input the network gets.
Metrics are still looking OK I guess : https://pasteboard.co/Hkc3lcQ.png
For completeness, here are the used hparams again (learning the lead section problem)

@registry.register_hparams
def transformer_base_wsum_v1():
hparams = transformer_base_v1()
hparams.max_input_seq_length = 500
hparams.max_target_seq_length = 0
hparams.max_length = 500
return hparams

Hi, I am trying to use the Transformer-Decoder-Only model from the "Generating Wikipedia..." paper for my own dataset.
I see that this thread points to instructions for generating the WikiSum dataset but I didn't see instructions for running the model as described in "Generating Wikipedia..." ?
Also, are there any guidelines on how I should format my dataset so that the T2T Transformer-Decoder-Only model can consume it?

Closing this bug as data generation now works. Thanks @f-lng for the bug finding!
For training, I believe these settings produce reasonable results. We usually trained on 8 GPUs or a TPU:
model = 'transformer'
hparams = "max_input_seq_length=$MAX_LEN"
hparams_set = 'transformer_base'
decode_hparams = 'beam_size=4,alpha=0.6,batch_size=1'


Versions 0.8.1 is better than 0.9.0

I use 0.8.1 because 0.9.0 has a couple of issues. First, it does not show itself when I'm in another fullscreen app. Second, it has dock icon which is always present. Will it be fixed in next versions?

what app for fullscreen?
dock icon will be configurable in next version

Any app on a mac if it's in fullscreen. Stretchly does not show the window with the break, I just hear sound from time to time.

Not happening on master branch so I guess that was some issue with electron version used on 0.9.0. I will close this once I release new version.

Thank you.


Flakyness in tests

Some times CI reports this error and usually a re-kick solve the issue, find the origin of the flakyness
---- test::test_multi_wallet stdout ----
thread 'test::test_multi_wallet' panicked at 'assertion failed: (left == right)
left: Amount(1.00000000 BTC),
right: Amount(0.00000000 BTC)', src/lib.rs:635:9

fixed in #75


Python (Microsoft Store edition) installation not recognized

Hello, I installed python from Microsoft Store.
Everything works fine, but VScode gives me this error.
How I can hide this error message?

Now I have to add below line in settings.json
"python.disableInstallationCheck": true

I think you should uninstall that python,then Google python and install it
from python's official website.And don't forget to add python to your PATH.
HAPPY PYTHONING!
On Sat, 25 Apr 2020, 16:18 Muhammad Hussein Ammari, <
<EMAIL_ADDRESS>wrote:

Hello, I installed python from Microsoft Store.
Everything works fine, but VScode gives me this error.
How I can hide this error message?
[image: image]
https://user-images.githubusercontent.com/23485114/80283442-d16b4380-872c-11ea-875c-85a9763ecdc4.png
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-python/issues/11413, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOUFKPAH75D2KOFVUAATCADROL5KTANCNFSM4MQZGFVA
.

No just delete the app.
On Sat, 25 Apr 2020, 16:41 Muhammad Hussein Ammari, <
<EMAIL_ADDRESS>wrote:

Now I have to add below line in settings.json
"python.disableInstallationCheck": true
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-python/issues/11413#issuecomment-619398047,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOUFKPGLE6YXQZTNXB6YQHTROMABVANCNFSM4MQZGFVA
.

Visual Studio Code says download Python from the Microsoft Store.

This is dupe of #10850. This has been fixed in master, fix will be available in the next release.


Allow public clients without a client secret

add PublicClient flag

This is something I would really like :)

done on dev


[CI] Update icx compiler to 2025.0

Description
Match changes occurring in oneapi-src/oneDAL#2958 which upgrades to the latest Intel DPC++ compiler.

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.
You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).
Checklist to comply with before moving PR from draft:
PR completeness and readability

[ ] I have reviewed my changes thoroughly before submitting this pull request.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
[ ] Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
[ ] I have added a respective label(s) to PR if I have a permission for that.
[ ] I have resolved any merge conflicts that might occur with the base branch.

Testing

[ ] I have run it locally and tested the changes extensively.
[ ] All CI jobs are green or I have provided justification why they aren't.
[ ] I have extended testing suite if new functionality was introduced in this PR.

Performance

[ ] I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
[ ] I have provided justification why performance has changed or why changes are not expected.
[ ] I have provided justification why quality metrics have changed or why changes are not expected.
[ ] I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

/azp run CI

/azp run CI

/azp run CI


SLM Under Different Navigation Field

I have Just Started Using this plugin With Netbox 3.7.5 for managing Licenses in my organisation. Everything looks good and it's a perfect fit for my needs.
I just have 2 Queries regarding this:

How I can Move it to a different Navigation Field? As of now, it's coming under Plugins. Is there any way that I can move it under some different or new field?

When In noebox we are not logged in it Shows the Plugin tab and SLM under this. Anyway to hide this

Hi @rudradevpal

I don't think you can adjust this via the GUI. Perhaps by adjusting the code, but this is a plugin after all, so it makes sense for us that it is under 'Plugins'. But suppose we will move it, if possible, what do you think would be a better place?

I think there is a permission check missing on the menu items for the plugin. Can you take a look at this @wkoot? I think a 'permissions' variable is missing on the 'PluginMenuItem' items in netbox_slm/navigation.py.

@MartVisser Thanks for the reply.

I guess a section called "Software" will be a better option


terraform destroy fails due to empty coalesce in output

Describe the Bug
Using terraform 0.12:

create a main.tf that uses this module
terraform init and apply
terraform destroy

eventually the following happens:
$ terraform destroy
...

  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

Error: Error in function call

  on .terraform/modules/terraform_state_backend/output.tf line 30, in output "dynamodb_table_id":
  30:     coalescelist(
  31:
  32:
  33:
    |----------------
    | aws_dynamodb_table.with_server_side_encryption is empty tuple
    | aws_dynamodb_table.without_server_side_encryption is empty tuple

Call to function "coalescelist" failed: no non-null arguments.

Expected Behavior
It should have finished properly. Looks like the coalesce is incorrect.
Fix
This is likely due to the new behavior of coalescelist() in terraform 0.12.
Add [""] to the coalesclist() call. I will try to submit a PR.

So upon further thought, adding the empty string list is a hack for backwards compat with terraform 0.11, because the 0.12 coalescelist behavior is correct: the function returns the first non-empty list so if they are all empty, it does not make sense to return anything and it should raise an exception.
The deeper issue is that terraform tries to render outputs as part of a destroy, which doesn't make sense in general and is likely to have other similar issues with undefined objects. See my comment https://github.com/hashicorp/terraform/issues/25390#issuecomment-649493926.

This issue seems resolved @schollii . If not, please comment the ticket again.
Thanks again for the contribution.


Snowfall may not be reported correctly

I think open-meteo has a separate variable for snowfall than precipitation. I'm noticing that the precipitation value seems way too low for when snowfall occurs in areas that (anecdotally) have received more.
I'll need to adjust for counting snowfall as well

So, there are separate variables for total precipitation vs fine-grained ones for rain, showers and snow:

I think the precipitation is largely dependent on what the weather models return, but I'll see if there's a better way of calculating this. Most of the time, the precipitation accumulation seems too low, like 0.01 to 0.02 inch, even for snowfall.


BeagleBone Space error

cannot copy extracted data for './usr/share/doc/libice-dev/changelog.gz' to '/usr/share/doc/libice-dev/changelog.gz.dpkg-new': failed to write (No space left on device)
Selecting previously unselected package libsm-dev:armhf.
Unpacking libsm-dev:armhf (from .../libsm-dev_2%3a1.2.1-2_armhf.deb) ...
dpkg: error processing /var/cache/apt/archives/libsm-dev_2%3a1.2.1-2_armhf.deb (--unpack):
 cannot copy extracted data for './usr/share/doc/libsm-dev/changelog.Debian.gz' to '/usr/share/doc/libsm-dev/changelog.Debian.gz.dpkg-new': failed to write (No space left on device)
Selecting previously unselected package libpixman-1-dev.
Unpacking libpixman-1-dev (from .../libpixman-1-dev_0.26.0-4+deb7u1_armhf.deb) ...
dpkg: unrecoverable fatal error, aborting:
 unable to flush /var/lib/dpkg/updates/tmp.i after padding: No space left on device
E: Sub-process /usr/bin/dpkg returned an error code (2)

After checking memory left we had about 165 MB left.

We changed to a new beagle bone and started from scratch. Now we get the following error hen we run the qr reading program:
Traceback (most recent call last):
File "qr_feed.py", line 48, in
pil_im = Image.open('buffer.png').convert('L')
File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 2295, in open
% (filename if filename else fp))
IOError: cannot identify image file 'buffer.png'

I think it doesn't have drivers installed.
It connected automatically before so I'm not sure why it didn't work this
time when it had internet access.
On Feb 2, 2016 9:51 PM, "BrettGoldbach"<EMAIL_ADDRESS>wrote:

We changed to a new beagle bone and started from scratch. Now we get the
following error hen we run the qr reading program:
Traceback (most recent call last):
File "qr_feed.py", line 48, in
pil_im = Image.open('buffer.png').convert('L')
File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 2295, in
open
% (filename if filename else fp))
IOError: cannot identify image file 'buffer.png'
—
Reply to this email directly or view it on GitHub
https://github.com/IEEERobotics/bot/issues/450#issuecomment-178969361.


What is an identity secret?

identitySecret: 'XXXXXXXXX/XXXXXXXXXXXXXXX/X=', //identity secret, needed to automatically confirm trade offers, must be used with confirmationInterval

That is the secret key which allows your bot to accept/send trade offers

But how do you find it?
Sent from myMail for iOS
Friday, 18 March 2016, 19:20 +0000 from<EMAIL_ADDRESS><EMAIL_ADDRESS>
That is the secret key which allows your bot to accept/send trade offers
—
You are receiving this because you authored the thread.
Reply to this email directly or  view it on GitHub

http://www.lmgtfy.com/?q=where+to+find+steam+identity+secret

@infam0us that doesn't help much considering the second result is this github repo

@infam0us or im blind lol :3

@CodeLuca Let me even click that for you :P it actually depends on where did you actually activated your 2FA. Where did you activated it?

@infam0us im using mobile auth?

@CodeLuca with you phone, right?
For IOS: http://forums.backpack.tf/index.php?/topic/45995-guide-how-to-get-your-shared-secret-from-ios-device-steam-mobile/
For Android: http://forums.backpack.tf/index.php?/topic/46354-guide-how-to-find-the-steam-identity-secret-on-an-android-phone/

thx

np m8

The identity secret is found on the generated file when you activate 2fa via parentbot.

@dragonbanshee inside the sentry or log?

inside the sentry or log?

@CodeLuca inside sentry

@infam0us how does one read a sentry file?

@CodeLuca I'm really sorry. It's not sentry. it is in [steam-username].2fa_info . If you don't have one, you didn't activated via parentbot.

could you direct me how to activate it?

Just logging in with parentbot will activate it automatically assuming it isn't activated anywhere else.

I dont seem to have these files inside my project directory when running my bot.

My end goal here is to get automatic trade confirmations when I send a trade.

@andrewda @dragonbanshee @infam0us

Apparently, your 2FA is activated somewhere else. If you don't know where you activated it, no one can help.

this is all of the stuff I do in my index.js file, which activates parentbot: https://gist.github.com/CodeLuca/44de80dfc35619357b26

@CodeLuca your 2FA problem is not about the code. It's about your steam account.

I can receive trades with the code I have written just auto confirming them that is the problem. You sure this is to do with the account?
Tuesday, 22 March 2016, 10:43 +0000 from<EMAIL_ADDRESS><EMAIL_ADDRESS>
@CodeLuca your 2FA problem is not about the code. It's about your steam account.
—
You are receiving this because you were mentioned.
Reply to this email directly or  view it on GitHub

@infam0us I know what 2FA is, I have enabled it and I have used it in other parts of the same project, I just need the identity secret to auto accept trades.

#17


Deprecate backbone.validation package on NPM in favor of backbone-validation

Just ran into the issue where an old tutorial was used and someone ran npm install backbone.validation instead of backbone-validation
NPM docs have some instructions for deprecating a package
https://docs.npmjs.com/cli/deprecate
npm deprecate backbone.validation@"< 0.7.2" "WARNING: This project has been renamed to backbone-validation. Install using backbone-validation instead."

backbone.validation was published by someone who forked this lib and published it under that different name https://github.com/blikblum/backbone.validation so there is nothing that @thedersen can do about that.


Are you using some sort of cache?

For some reason, when I add a new file in my folder (note that it is inside a node_modules folder), on rebuild it's not detected. For some reason I need to change the glob (for instance switch between
whatever/* and whatever/*.js).
So is there some sort of cache in your implementation that could cause that?
The bug happens with webpack and babel on my project. I don't really have time to create a standalone reproduce of the bug, but I just like to understand what might be causing it.

The bug happens with webpack and babel on my project.

Either of those may be using a cache. As far as I know there's currently no way for this plugin to signal that the file should never be cached.
Re-saving the file that contains the glob may be sufficient, without actually having to make any changes. Sometimes it also helps to restart the webpack / Babel CLI watchers.

yes it seems react-natives metro bundler also caches the modules resolved by this plugin and does not refresh them until resetting the cache. Thats kind of sad. It should not cache those dynamic modules... maybe it can be convinced to not cache certain module patterns.


Mathematica solves this but not Rubi

1/(1 + 2 x + x^3 - x^4)
Integrate[1/(1 + 2 x + x^3 - x^4), x] // Normal // ToRadicals // Simplify
gives sensible extensive explicit answer in elementary functions.
The same is with
1/(1 + 2 x - x^4)
1/(x^4 + 2 x + 4)
Mathematica easily solves this: (x^4 + x^3)/(x^4 - x^3)
with result x + 2 Log[-1 + x] but Rubi cannot. The same for  (x^4 + x^3)/(x^4 - x^2)  and (x^4 + x^2)/(x^4 - x^3)

Sorry, Rubi can't integrate rational functions involving quartics in the denominator, unless they can be factored.
The currently available Rubi 4.16.1 easily integrates (x^4 + x^3)/(x^4 - x^3), (x^4 + x^3)/(x^4 - x^2) and (x^4 + x^2)/(x^4 - x^3).
Albert


firebase auth not working in react-native when remote debugging is on

Steps to reproduce

npm install
react-native run-android

turn on remote debugging
Go to signin page and do a facebook sign in
Expected Behavior
Should sign in and go to home page
Actual Behavior
Nothing happens

Which devices? It works for me both in simulator and real device.

I've tried on both Genymotion(2.7.2) with image (Google Nexus 4 - 4.1.1 - API 16 - 768x1280 - 4.1.1) and my device (oneplus one).
I ran the app as it is, didn't change anything.
On my device, adb logcat did have an exception after I press the facebook login button, not sure if it's related:
8-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: Exception during service
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: com.facebook.http.protocol.ApiException: [code] 404 [message]: Key hash ATsIy/k6H7uzdokoI/w28hs5dFs= does not match any stored key hashes. (404) [extra]: null
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.protocol.ApiResponseChecker.a()
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.protocol.ApiResponse.j(:247211)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.platform.auth.server.AuthorizeAppMethod.a(:740421)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.protocol.ApiResponseHandler.handleResponse(:247416)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.common.FbHttpRequestExecutorAdapter.a()
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.common.FbHttpRequestExecutorAdapter.a(:243461)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.http.common.PriorityRequestEngine$ExecuteRequestsRunnable.run(:428492)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:423)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at X$id.run(:51811)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at com.facebook.common.executors.DefaultConstrainedListeningExecutorService$Worker.run(:96220)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at X$if.run(:51823)
08-10 00:17:31.177   777  4921 W fb4a(:):BlueServiceQueue: 	at java.lang.Thread.run(Thread.java:818)
Am I missing a config or something?

Try gulp android since it creates initialState with messages etc. Otherwise, I don't know.

gulp android didn't work unfortunately.
I tested on another Mac. Setup up react-native and Android SDK, clone este.js, build and gulp android. It had the same issue.

So it looks like Firebase errors and not an Este, so please use this unofficial still used by Firebase team issues https://github.com/steida/firebase/issues

I narrowed it down to one commit<12110152d70d087d142c1fdd1d62a1db4edefa56>. The change that broke it was the update of react-native to 0.31. If I switch back to 0.30, it works. I tried updating firebase 3.2.1, but that didn't fix it.


Reverted typescript to fix angular build.

Coverage remained the same at 94.54% when pulling d351060cde971cec31adbb82d05fb8108fae5610 on dev into 59d9c75cf304bcf92d790409dc3c60a7e6e6ae71 on master.


(Fixes #208)Add MongoDB integration persister

This PR adds a new MongoDB persister to the project, allowing state data to be saved and loaded from a MongoDB database. The implementation follows the existing structure of the Redis persister for consistency.
Fixes #208.
Changes

Added MongoDBPersister class to handle saving and loading state data to/from MongoDB.
Implemented methods: save, load, list_app_ids, and helper method create_key.
Added test_mongodb_persister.py to test the functionality of MongoDBPersister.

How I tested this
Created unit tests in test_mongodb_persister.py to verify the correct functionality of save, load, and list_app_ids methods.
Mocked MongoDB interactions to ensure tests do not depend on a live MongoDB instance.
Notes
Updated created_at to use timezone-aware datetime objects.
Checklist

[x] PR has an informative and human-readable title (this will be pulled into the release notes)
[x] Changes are limited to a single goal (no scope creep)
[x] Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
[x] Any change in functionality is tested
[x] New functions are documented (with a description, list of inputs, and expected output)
[x] Placeholder code is flagged / future TODOs are captured in comments
[x] Project documentation has been updated if adding/changing functionality.

If there is something which I missed or anything. please let me know.

I'll merge this tomorrow my time -- thanks @NandaniThakur !

I'll merge this tomorrow my time -- thanks @NandaniThakur !

As this was my first open source, It was quite fun & exciting to do. thank you for your guidance and help in completing this.
I'll look forward to the merging of PR. Thanks @skrawcz!!!


./setup.sh does not install all the prerequisets

Expected Behavior
Running ./setup.sh install all required python modules
Current Behavior
Loading binding c_transformers install ON
Traceback (most recent call last):
File "/home/gpt4all/gpt4all-ui/api/init.py", line 218, in _rebuild_model
self.binding = self.load_binding(self.config["binding"], install=True)
File "/home/gpt4all/gpt4all-ui/api/init.py", line 158, in load_binding
binding_module = loader.load_module()
File "", line 548, in _check_name_wrapper
File "", line 1063, in load_module
File "", line 888, in load_module
File "", line 290, in _load_module_shim
File "", line 719, in _load
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/gpt4all/gpt4all-ui/bindings/c_transformers/init.py", line 18, in
from ctransformers import AutoModelForCausalLM
ModuleNotFoundError: No module named 'ctransformers'
Couldn't build binding
Steps to Reproduce
Please provide detailed steps to reproduce the issue.

Log into Debian Windows Subsystem for Linux with python3.10 and python3.11 installed
Run ./setup.sh
Browse to <IP_ADDRESS>:9600
Click on Settings
Expand Bindings zoo
Click on C Transformers
Attempt a conversation

Possible Solution
Add an exception handler to install the missing python modules "/home/gpt4all/gpt4all-ui/api/init.py" for line 158 load_binding, if the module is missing or perhaps add the missing modules to requirements.txt
Context
Debian VM with both python3.10 and 3.11

go to the bindings folder, then to c_transformers and remove .installed then re-run. It should install all required things for that binding. the setup installs a minimal configuration then when you first run a binding it installs its requirements. And I think someone forgot to remove .installed from the git before committing and that's why it si not working for you. This file indicates that the binding is already installed.
I'm not reparing this as I am working on a new branch that has already fixed this issue. It will be merged soon.

I've found for deb/Ubuntu, build-essential isn't a prerequisite. It should be as it isn't installed by default on (many) installations.
That is--if you haven't already fixed this. I appreciate your contribution.

Hi. I'm preparing a new video on the new conda based installation. It installs everything you need with cuda and build tools.
I didn't yet test the linux version. Coded it blindly since i have no linux machines this time around.


Opening a ruruku page without token has no error message

If a user opens a ruruku participation page without a token, the only error message is in the console upon connection. We should instead hide the username field and show an appropriate message.

fixed in #20


Expanding the top of a window against the top of the screen makes it grow downwards

Description

Put window against the top of the screen.

Click and hold.
Drag up.
It just goes.

Doing the same downwards does not replicate the same behavior.
https://user-images.githubusercontent.com/10968691/135669208-c07e0f73-091e-415d-963a-9300c7c5ae13.mp4

Same goes for left side. There is some other weird behaviour like the window moving left when you try to make it smaller from the right and it has reached the minimum size, iirc.

Related to https://github.com/space-wizards/space-station-14/issues/3724


[OCCM] Add annotation for creating health monitor

The binaries affected:

[ ] All
[x] openstack-cloud-controller-manager(OCCM)
[ ] cinder-csi-plugin
[ ] k8s-keystone-auth
[ ] client-keystone-auth
[ ] octavia-ingress-controller
[ ] manila-csi-plugin
[ ] manila-provisioner
[ ] magnum-auto-healer
[ ] barbican-kms-plugin

What this PR does / why we need it:
Add service annotation loadbalancer.openstack.org/enable-health-monitor to define whether or not to create health monitor for the load balancer pool, if not specified, use 'create-monitor' config. The health monitor can be created or deleted dynamically.
The scenaro to use this annotation is to deal with the pod that exposes TCP via SNI (e.g. Envoy) and is performing client certificate validation, in this case the health monitor created will fail. We need provide the capability to customize.
Which issue this PR fixes:
fixes #
Special notes for reviewers:

Release note:

openstack-cloud-controller-manager: Added a service annotation `loadbalancer.openstack.org/enable-health-monitor` to define whether or not to create health monitor for the load balancer pool.

/test cloud-provider-openstack-acceptance-test-csi-manila

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.15

/test pull-cloud-provider-openstack-test

/test cloud-provider-openstack-acceptance-test-lb-octavia

/lgtm

/approve


Check that license stuff is in order

@massie could you make sure that everything here is kosher?  I just copied the header we use in ADAM.

poke @massie.

Looking now.

Looks good. :+1:

Sweet, thanks @massie!


purism/librem5r4: linuxPackages_librem5: 6.4.14-librem5 -> 6.5.6-librem5

Description of changes
Update the linux kernel to the latest release on https://source.puri.sm/Librem5/linux
Things done

[X] Tested the changes in your own NixOS Configuration
[X] Tested the changes end-to-end by using your fork of nixos-hardware and
importing it via <nixos-hardware> or Flake input

@mergify queue


Will it be good to use compiled validate function as a schema in the schemas option of validator(schema, { schemas })?

I am extending json-schema by implementing $import.
If compiled validate function can be used as a schema in the schemas option of validator(schema, { schemas }),
my implementation would be easy and no hack.
My repo is at: https://github.com/xieyuheng/json-compact-schema

I I found that schema in schemas can refer to each other.
My problem is solved.


Allow select2_js to be list

In certain cases such as breakage to select2 like the one below, one can have a patch by loading another js.
https://github.com/select2/select2/issues/5993

Hi @pickfire,
Thanks for reaching out. Interesting idea. Your implementation would change the API, but be backwards compatible. However, we might want to add deprecation here. I don't want to keep legacy code or type casting in the code forever. Deprecation, in the next major version, should be fine.
I am curious though, what is your use case? What other JS do you need to inject his way? If it's unrelated, I'd recommend writing your own custom subclass to inject other form media.
Cheers!
Joe

I am curious though, what is your use case? What other JS do you need to inject his way? If it's unrelated, I'd recommend writing your own custom subclass to inject other form media.

One js is to load the original select2 javascript, another js is to patch select2 bug to work with never version of jquery.


[WIP] Add support for arrays in uniforms

Still needs tests and perhaps a rework overall

Updated with suggested changes


Uncaught Error: Module did not self-register.

[Enter steps to reproduce:]

...
...

Atom: 1.46.0 x64
Electron: 4.2.7
OS: Mac OS X 10.14.6
Thrown From: Atom Core
Stack Trace
Uncaught Error: Module did not self-register.
At ELECTRON_ASAR.js:160

Error: Module did not self-register.
    at process.module.(anonymous function) [as dlopen] (ELECTRON_ASAR.js:160:31)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:722:18)
    at Object.module.(anonymous function) [as .node] (ELECTRON_ASAR.js:169:18)
    at Module.load (internal/modules/cjs/loader.js:602:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:541:12)
    at Function.Module._load (internal/modules/cjs/loader.js:533:3)
    at Module.require (/app.asar/static/index.js:72:46)
    at require (internal/modules/cjs/helpers.js:20:18)
    at customRequire (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:1:695134)
    at get_KeyboardLayoutManager (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:14:2778162)
    at get_manager (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:14:2778264)
    at Object.getCurrentKeyboardLayout (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:14:2778390)
    at e.keystrokeForKeyboardEvent (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:14:1081110)
    at KeymapManager.keystrokeForKeyboardEvent (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:11:1253631)
    at KeymapManager.handleKeyboardEvent (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:11:1251616)
    at WindowEventHandler.handleDocumentKeyEvent (/Applications/Atom.app/Contents/Resources/app/static/<embedded>:11:284744)

Commands
     -0:29.8.0 core:close (input.hidden-input)
  8x -0:10.9.0 core:copy (input.hidden-input)

Non-Core Packages
atom-material-syntax 1.0.8
autocomplete-python 1.16.0
busy-signal 2.0.1
intentions 1.1.5
language-latex 1.2.0
latex 0.50.2
lines 0.13.1
linter 2.3.1
linter-flake8 2.4.0
linter-ui-default 1.8.0
pretty-json 2.1.2

Thanks for contributing!
We noticed that this looks like a duplicate of https://github.com/atom/atom/issues/20049 so you can subscribe there for updates if you'd like.
As mentioned in https://github.com/atom/atom/issues/20049#issuecomment-546646559, it would be helpful to know if you still see the error after quitting/restarting Atom.

Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Implement slices

This PR adds basic implementation of slices to be extended in next PRs

What happened here? Why does this PR contain 30 commits from last week?

This looks nice, but I'm scared of giving it an approve because of how the history looks like. It's not too much code so maybe let's close this PR and open a nice one with just a couple of commits


Modify the subpx util to wrap around text layer instead of inside

This is a bug fix for annotations, because text selection serializes
differently with/out the subpx layer when using .crocodoc-page-text as
the reference element.

Verified that @lakenen has signed the CLA. Thanks for the pull request!


Sugar subscribe()

Currently if you want to access previous value of a subscribed signal you have to
$signal.subscribe((v) {
      final previous = $signal.previousValue;
      doSomething(previous) then ...
});

Signature would have been much better with subscribe(previous, current) for chaining.
$signal.subscribe((previous, current) {
     doSomething(previous)
});

Can be expressed with elegance when non required
$signal.subscribe((_, current) {

});

Maybe a real consideration for people pushing Signals it a lot of sugar to not repeat the var and less error prone.
As a general remark Signals should get inspired (copy without questions) functional signatures and move away from oop.
equalityCheck can be hacked in the meantime, it has the right signature but come on ;D

This could be done via a set of extension methods without having to update the type signature!

The idea never touched my mind
Thanks a lot for the feedback!
I'm digging about something more general and also prehook so we can act pre and post update.

Signal hooks for sure. I think that would be possible in general and would love to add support for it.
It could be a wrapper signal that adds the functionality just like a change stack too

Most of the logic is already there!
Signal handle transmission but you just need confirmation/infirmation upfront and lifecycle of previous signal. When you switch to a new signal value you may have to handle resources not in the signal itself but the previous value. It should be handled before managing effects of the new value.


Add2App functionality needs to have some test infrastructure

Without infrastructure to test flutter create -t module for example simple fixes (like https://github.com/flutter/flutter/pull/23817) have to be manually tested.
cc @matthew-carroll

There are tooling tests now in devicelab tasks now.


MINIFICPP-67: Merge Content processor

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[ ] Does your PR title start with MINIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

[ ] Is your initial contribution a single, squashed commit?

For code changes:

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file?
[ ] If applicable, have you updated the NOTICE file?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Please review and provide feedback.

@benqiu2016 I"m currently reviewing, but have yet to finish it, at which point the comments will open up.

@apiri please review the same while Marc is out. I addressed his comments

@phrocker @apiri please approve the same, i would like to merge that before i add the compression/tar support for merge content.

@minifirocks Can you rebase against master? Your commits have unrelated content.

@phrocker rebased

@phrocker @apiri please let me know whether it can be merged to master.

@minifirocks I did have a question that popped up after I hit approve....with the header and footer you're serializing the data directly. Did you have any issues when opening those merged content files in NiFi after it was sent via Site To Site?

@phrocker i did not send the file over site2site, i save to a file and i can open it for read OK. the sending of these flowfile is the same procedure that we use to send normal flow files.

@minifirocks What did you use to open it?  The procedure is the same, but the interpretation of that data is not. Seems that we need to at least verify that NiFi can interpret what we send it.

@phrocker i tie the merge processor to a put file processor to save the content

@phrocker let me test the NiFi site2site to double check

@phrocker test the site2site, NiFi receive the right header/footer/demarcator

@phrocker @apiri please take a look and see whether you can merge the PR to master. i tested the site2site as above.

@minifirocks thanks! I'll merge to master at some point today

@phrocker thanks for the help


Documentation failing.

I'm guessing this is an issue with babel-core and apparently is only affecting depending packages, since the documentation is generated fine on this project.

After further investigation, this seems to be a problem with jsdoc-to-markdown. I'm starting to think that I'll have get it out of the way and use de jsdoc parser directly. It implies extending the implementation for the template, so this is not a small task.
A solution while this is implemented is to set the build script with commenting on, and set the docs script to use the out directory instead of src, but right now that's not implemented, so I'll add a configuration in the meantime.

Example fix on package.json
{
    ...
    "@gik/npm": {
        "babel": {
            "comments": true
        },
        "jsdoc": {
            "files": "./out/**/*.js"
        }
    }
}


Fix casing of dynamic tags

Description
Fixes #2483
Related #1269: I'm not sure that's fully addressed with the PR
There's a comment by @siku2 about preserving capitalization, but that doesn't seem to be what's implemented, at least not everywhere.
Checklist

[x] I have run cargo make pr-flow
[x] I have reviewed my own code
[x] I have added tests

After further playing around with it, and forgetting to sometimes use the dynamic tag form, I've decided to add a lint. Only shows up on nightly, like the other lints, but I think it's better than nothing.

After further playing around with it, and forgetting to sometimes use the dynamic tag form, I've decided to add a lint. Only shows up on nightly, like the other lints, but I think it's better than nothing.

Even better! :)


Delivery Tax disappears when clicking on delivery

Expected behavior:
GIVEN Tax Mode is enabled
AND Show Tax Breakdown in Storefront is enabled
AND Tax Delivery Charge
WHEN clicking on "Pick-up" in the cart box the field "Delivery Tax:" should disappear and then clicking back on "Delivery" the "Delivery Tax:" field should REAPPEAR
Actual behavior:
Clicking back on delivery after clicking "Pick-up" OR having the page refresh whilst "Pick-up" is still selected, the "Delivery Tax:" field does not appear
Reproduce steps:
Enable Tax Mode, Show Tax Breakdown in Storefront and Tax Delivery Charge in the admin panel
Set a delivery charge of 'x' amount when cart exceeds 'y' amount
Load storefront and add items to basket and ensure you go over the 'y' amount to get the 'x' delivery charge
Click pick-up and then click back on delivery and the field will disappear
Version:
(TastyIgniter version number)
Additional Information:
(Additional information, if any)

See if the commit fixes the issue, not sure if I understood the issue right.

This issue is fairly old and there hasn't been much activity on it. Closing, but please reopen if it still occurs.