Order Rule incorrectly warns on no empty line for mutli line imports

If you have the following in your .eslintrc:
import/order: 2
single line imports work great, but if you have mutliline imports due to max width, order incorrectly warns.
For example:
import {
  foo, bar,
  buzz,
} from './foo';
import cheese from './cheese';

You get an invalid warning. It should only error when there is an unnecessary newline
There are also invalid warnings on comments such as:
import foo from './foo';
// Comment about import below
import bar from './bar';

I don't believe either of these should trigger an error

Ah I just saw #339. @singles does your PR also allow comments on their own line between import groups?
Looks like multi line imports are fixed in that PR

@mjrussell This works fine (newlines-between set to never)
var path = require('path');
//some comment here
require('something-else');
var _ = require('lodash');

Code iterates over lines as a strings, trims those and if after trimming it has length === 0, then it's considered empty. In case above comment will not be empty and therefore error in this case will not be reported.
Yes, multiline imports were fixed in mentioned PR, sorry missing that earlier :)

Cool thanks will test and validate when the new version is released

@mjrussell Did new version 1.8.1 solved your problem? If yes, we could close this ticket.

@singles I'll check and report back. Thanks for the note about the new release

It does indeed solve the issue, closing. Thanks!


Reuses skipped checkrun, as we can't remove it

Changes
Reuses skipped checkrun, as we can't remove it
skipped runs were not getting removed a authorised user adds /ok-to-test
on a pull req from a new user, this is to fix that case.
we cannot delete a checkrun so we reuse it for the pipelinerun, when the
CI is triggered later.
as we execute pipelineruns parallely, to make sure only one pipelinerun uses that
checkrunid we use mutex to store that checkrunid.

Submitter Checklist

[x] ♽  Run make test lint before submitting a PR (ie: with pre-commit, no need to waste CPU cycle on CI
[x] 📖 If you are adding a user facing feature or make a change of the behavior, please verify that you have documented it
[x] 🧪 100% coverage is not a target but most of the time we would rather have a unit test if you make a code change.
[x] 🎁 If that's something that is possible to do please ensure to check if we can add a e2e test.
[x] 🔎 If there is a flakiness in the CI tests then don't necessary ignore it, better get the flakyness fixed before merging or if that's not possible there is a good reason to bypass it. (token rate limitation may be a good reason to skip).

good idea 👍🏻

how hard is it to write an end to end test for this? TestGithubPullRequestOkToTest talks to the controller directly to have a non authorized user sending a payload, may be we can detect check runs in there if it was skipped and then reused

how hard is it to write an end to end test for this? TestGithubPullRequestOkToTest talks to the controller directly to have a non authorized user sending a payload, may be we can detect check runs in there if it was skipped and then reused

not sure.. 🤔 how this would be possible
after creating PR we want an existing checkrun.. and PAC execute the pull req event, it will override it


feat(cli): lacework policy delete

Introducing the ability to delete policies
ALLY-449

Needs rebase

Should be good.

Your GIF game is strong @afiune


Could not autowire field: private springfox.documentation.swagger2.mappers.ServiceModelToSwagger2Mapper

To study the architecture of Springfox, I've downloaded zip file of Springfox 2.0.1 and imported the package into Eclipse. After all the configuration work, however, what the error log on startup of the server says is below:
Error creating bean with name 'swagger2Controller': Injection of autowired dependen
cies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private springfox.docume
ntation.swagger2.mappers.ServiceModelToSwagger2Mapper springfox.documentation.swagger2.web.Swagger2Controller.mapper; nested exception is or
g.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [springfox.documentation.swagger2.mappers.ServiceM
odelToSwagger2Mapper] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency a
nnotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}

I have done with adding the package into the buildpath of a web project which will be auto-documented by Springfox 2.0.1 and configuring it using a class annotated '@EnableSwagger2'. It keeps showing up every trial on different sample projects. Maybe I am missing out a specific configuration or something. Pleeeease help me out of here. I really want to utilize Springfox on my project. I appreciate your help.

@choiapril6 I'd recommend downloading the 2.3.0 version instead.

Thanks for quick response! I will give it a try.

I did it with v2.3.0, but still am having same issue.

I am using Eclipse gradle plugin. So I imported Springfox 2.3.0 using the plugin.

—
Reply to this email directly or view it on GitHub.

Could you share your repo perhaps. It seems like a problem in your environment. Have you looked at the springfox-demos project?

The project on which Springfox is not working is below.
https://github.com/choiapril6/jwp-spring.git
I will look at springfox-demos too. Thanks.

omg I can't figure out why it keeps missing ServiceModelToSwagger2Mapper. I did annotate the class as Component but still failed. Anybody else with this same error?

It doesn't appear like you've added the following dependencies in your project
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.3.0</version>
        </dependency>
        <!-- Optional -->
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.3.0</version>
        </dependency>

Thanks for the answer. However, as I mentioned above, I'm not adding
dependency, but I downloaded full package of Springfox and add them to
build path instead. The reason why I am doing like this is that I want to
add some feature such as reading specific annotation that I made up. That
is why there's no springfox dependency in my pom.xml.
On Mon, Dec 14, 2015 at 12:27 AM, Dilip Krishnan<EMAIL_ADDRESS>wrote:

It doesn't appear like you've added the following dependencies in your
project
    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger2</artifactId>
        <version>2.3.0</version>
    </dependency>
    <!-- Optional -->
    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger-ui</artifactId>
        <version>2.3.0</version>
    </dependency>

—
Reply to this email directly or view it on GitHub
https://github.com/springfox/springfox/issues/1088#issuecomment-164268012
.

I really don't understand what you're trying to do.🤔
If your project is managed by maven it takes care of all the dependencies; even transitive dependencies. For eg springfox-swagger2 has other dependencies that are required. If your doing something half using maven and half build path, u need to make sure u have all those dependencies as well.
Sorry I'm not being of much help, but I think ur problem will be solved if u use maven as expected.

No problem:) Actually you are helping me.
Agree with your words. Half dependency half build path would not work out. But, I've downloaded and add all the projects including springfox-swagger2 into build path from the link(https://github.com/springfox/springfox/archive/2.3.0.zip). Is this way of building the environment a problem too?
Thanks for your patient help.

he reason why I am doing like this is that I want to add some feature such as reading specific annotation that I made up. So, I need to try add some code lines into Springfox source code.

If you're interested is in adding support of an annotation being processed you should just considering developing a plugin implementation and add that as a bean. The plugin architecture will take care of using your plugin implementation.
I don't think the hackery involved in using the build path etc. is warranted nor maintainable and definitely not supportable.
If you have a specific use-case I could provide you the guidance.

Thank you so much. I will elaborate the case for you.

The root cause for getting "No qualifying bean of type [springfox.documentation.swagger2.mappers.ServiceModelToSwagger2Mapper]" is that the ServiceModelToSwagger2MapperImpl has to be generated using gradle generateMappers and then add the springfox-swagger2/generated-src/main-folder to the source folders. Then the ServiceModelToSwagger2MapperImpl can be found on the classpath.


[Improvement]: Add UT for dashboard api

Search before asking

[X] I have searched in the issues and found no similar issues.

What would you like to be improved?
Currently, all dashboard APIs do not have unit tests, and we hope for two improvements.
First, modify the code of the APIs to make it easy to write unit tests.
Second, add unit tests to various modules.
How should we improve?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Subtasks

[ ] #2120
[ ] #2121
[ ] #2122
[ ] #2123
[ ] #2124
[ ] #2125
[ ] #2126
[ ] #2127
[ ] #2128

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @GOODBOY008, can I help complete these tasks?

@Jiabao-Sun It's very well to collaborate with you, You can comment at subtask and I will assign it to you.


ABE section is unclear and has duplicate wording

Please explain the ABE feature with better examples.
Please review duplicate wording:
\DFSServer\users\contosouser1 => \SA.file.core.windows.net\contosouser1 \DFSServer\users\contosouser1 => \SA.file.core.windows.net\users\contosouser1

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: af9042a5-da6a-fa01-773b-bcebf11bc9a7
Version Independent ID: e62ddb63-5b60-e187-2a6d-e7db00e3246e
Content: How to use DFS-N with Azure Files
Content Source: articles/storage/files/files-manage-namespaces.md
Service: storage
Sub-service: files
GitHub Login: @khdownie
Microsoft Alias: kendownie

@tony1111111111111
Thanks for your feedback! We will investigate and update as appropriate.


Piano: Extremely Simple, Single-server PIR with Sublinear Server Computation

Piano: Extremely Simple, Single-server PIR with Sublinear Server Computation

I have read the CLA Document and I hereby sign the CLA


Reordering tags

Hi guys,
First of all thank you for this amazing tool.
I tested the gcr-clear-cli locally with dry-run flag and I noticed something weird:

After running gcr-cleaner, all tags were reordered because the Uploaded timestamp was updated.
Unfortunately I don't have a screenshot of the order before running gcr-cleaner :(
Is this an expected behaviour?
Can I prevent this to happen?
To be sure I tried on another repo and the behaviour was not the same, the order of tags remained the same]

Some additional info:

running gcr-cleaner from main branch
running against private Google Container Registry eu.gcr.io
gcloud versions: Google Cloud SDK 358.0.0; beta 2021.09.17; bq 2.0.71; core 2021.09.17; gsutil 4.68

I did another test and I noticed that if in the repo there are "numeric tags", gcr-cleaner seems to perform a weird "sorting/reorder".
here you see the situation BEFORE

and here AFTER running gcr-cleaner

and here gcr-cleaner logs

go run ./cmd/gcr-cleaner-cli/main.go -repo eu.gcr.io/*** -dry-run
WARNING: Running in dry run mode! Nothing will actually be cleaned.eu.gcr.io/***: deleting refs since 2021-09-29 08:32:46.018935 +0000 UTC
[dry-run] would delete: eu.gcr.io/***@sha256:89bf4ea21946c3cbfb410d946b16dbe25036502ae70ac4a4239c7c350ac126d2
[dry-run] would delete: eu.gcr.io/***@sha256:dcf92545839cac45f0d2dd262085f562c43c87b7156c45bebad6b03b2d690985
[dry-run] would delete: eu.gcr.io/***@sha256:bb16860b9c44c88dd4fe57979f7d70fd69a47bb9824946d125e27730548be02f
[dry-run] would delete: eu.gcr.io/***@sha256:2eead905735fc4cc8f473eb06b237d6b7db4b00623f7fb59e4f5cf00280606bc
eu.gcr.io/***: successfully deleted 4 refs

is this behaviour expected?

The "updated_at" timestamp is changing, which is why you're seeing the items move. If you sort by tag (click on the header) or another column, it should remain consistent. As for why the timestamp is being updated, I'm not entirely sure. I think that's just how the API works.

@sethvargo ok probably I got my mistake.
In the test done in the previous comment, I added the tags "0.0.1" and "1.0.0" on purpose right before running gcr-cleaner, to see the behaviour of the tool with different kind of tag. After adding them, I didn't refresh the page and I thought the reordering was due to the tool but I was wrong!
Sorry for this issue and thanks for your patience!


Multiple routes for the same message

Hi, can a message be matched with multiple routes and hence be routed to multiple endpoints? Or, will a message only be sent to the endpoint for the matched route with the highest priority? Thanks

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ad3731a8-f0df-f1a1-d1a1-e89a769a0908
Version Independent ID: 50a5190e-eafa-7dde-c18f-be9505b00a0a
Content: Deploy module & routes with deployment manifests - Azure IoT Edge
Content Source: articles/iot-edge/module-composition.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

@ningweinw Thank you for the question. We will review and get back to you.

@ningweinw Messages are queued up based on their endpoints. All priority 0 messages targeting a specific endpoint will be processed before any priority 1 messages targeting the same endpoint are processed, and down the line. If multiple routes for the same endpoint have the same priority, their messages will be processed in a first-come-first-served basis. If no priority is specified, the route is assigned to the lowest priority.
Does that answer your query? Let us know if you need further help.

Hello @ningweinw Please let us know if you need further help in this matter.

Hi
Thank you for your response, that answers my question
Best Regards
Ning Wei
On Fri, Nov 6, 2020 at 9:20 AM SatishBoddu<EMAIL_ADDRESS>wrote:

Hello @ningweinw https://github.com/ningweinw Please let us know if you
need further help in this matter.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/65330#issuecomment-722745281,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKLZ7WU656IFFSPCAZUDXVDSONFMNANCNFSM4TFYNUKA
.

@ningweinw Thanks for the confirmation.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


QPY invalid payload generation via compose() or QuantumCircuit.control()

Environment

Qiskit Terra version: 0.23.2
Python version: 3.10
Operating system: Linux

What is happening?
Running compose() with inplace=True from an input generated with QuantumCircuit.control() leads to a circuit that when qpy serialized is not load-able. This points to an internal state that doesn't match the actual data of the instruction object. I expect the mismatch is caused by the number of arguments or qubits the gate reported in qc which is incorrect
The specific failure in this case is:
Traceback (most recent call last):
  File "/tmp/test_qpy_roundtrip.py", line 17, in <module>
    new_qc = load(fd)[0]
  File "/tmp/foo/lib/python3.10/site-packages/qiskit/qpy/interface.py", line 269, in load
    loader(
  File "/tmp/foo/lib/python3.10/site-packages/qiskit/qpy/binary_io/circuits.py", line 905, in read_circuit
    _read_instruction(file_obj, circ, out_registers, custom_operations, version, vectors)
  File "/tmp/foo/lib/python3.10/site-packages/qiskit/qpy/binary_io/circuits.py", line 161, in _read_instruction
    struct.unpack(
struct.error: unpack requires a buffer of 33 bytes

How can we reproduce the issue?
import io

from qiskit.circuit.random import random_circuit
import numpy as np
from qiskit import QuantumCircuit
from qiskit.qpy import dump, load

qc0 = random_circuit(2, 2, seed=1).decompose(reps=1)
qc1 = random_circuit(2, 2, seed=1).decompose(reps=1)
qc = QuantumCircuit(3)
qc.compose(qc0.control(1), [0,1,2], inplace=True)
qc.compose(qc1.control(1), [0,1,2], inplace=True)

with io.BytesIO() as fd:
    dump(qc, fd)
    fd.seek(0)
    new_qc = load(fd)[0]

assert qc == new_qc

What should happen?
This should not error during the load() call
Any suggestions?
I believe something about the compose call is corrupting the internal state of the circuit which is leading to a QPY payload that has a mismatch between a size

For anyone experiencing this you can work around this failure by changing:
qc.compose(qc0.control(1), [0,1,2], inplace=True)
qc.compose(qc1.control(1), [0,1,2], inplace=True)

to
qc.append(qc0.control(1), [0,1,2])
qc.append(qc1.control(1), [0,1,2])

I think something's a bit odd about QuantumCircuit.control: it constructs a gate, from the circuit, controls that, and adds the resulting gate to a circuit.  That means that when you compose it onto a circuit, it's still a wrapped custom instruction rather than having been properly inlined.
If I change QuantumCircuit.control to just return circuit_to_gate(self).control(...).definition, the QPY stuff works fine and that code makes more sense.  That said, it's not necessarily the correct fix for here (unless there's an actual bug in QuantumCircuit.control), because the current form should still be producing a valid circuit that roundtrips through QPY.

Hmm, that seems odd qpy shouldn't care about it being wrapped in a custom instruction. I wonder if it's the same bug as https://github.com/Qiskit/qiskit-terra/issues/8941 where the controlled gates are ending up with the same names and that's causing issues.

I'm suspicious that there's a mistake in the recursive handling, when there's multiple custom instructions that all contain other custom instructions.  I'm fairly confident the issue happens during the QPY dump, not the read. I instrumented write_circuit with a print(circuit.name), and it shows an asymmetry between how it handles the two gates - it shows that it touches qc.data[1].operation._definition and qc.data[1].operation.base_gate.definition, but only one of those two things for qc.data[0].

Hi all, thanks for looking into this issue !
The workaround proposed above (with inplace=False)  will create two empty circuits though right ?
From what I could understand, when qpy dumps the base circuit (here: https://github.com/Qiskit/qiskit-terra/blob/16f6adb310719619f5cc07d314a95f12d6ea07c4/qiskit/qpy/binary_io/circuits.py#L649) something might be going wrong with the format. So when reading the .qpy file, the reader manages to read the first base circuit but fails at the second.
Thanks again !

The key to the workaround I suggested in https://github.com/Qiskit/qiskit-terra/issues/9746#issuecomment-1458534286 is that it's using the  append() method instead of compose() and that is always done in place. So taking the OP code it returns a circuit that looks like:
     ┌───────────────┐┌───────────────┐
q_0: ┤0              ├┤0              ├
     │               ││               │
q_1: ┤1 c_circuit-88 ├┤1 c_circuit-91 ├
     │               ││               │
q_2: ┤2              ├┤2              ├
     └───────────────┘└───────────────┘

which decomposed looks like:
q_0: ───────■──────────────■───────
     ┌──────┴──────┐┌──────┴──────┐
q_1: ┤0            ├┤0            ├
     │  circuit-88 ││  circuit-91 │
q_2: ┤1            ├┤1            ├
     └─────────────┘└─────────────┘

and another layer deeper is:
q_0: ──■───■──────────■───■─────────■───■──────────■───■──────────■───■───────»
     ┌─┴─┐ │P(-π/4)   │   │         │   │        ┌─┴─┐ │P(-π/4)   │   │       »
q_1: ┤ X ├─■──────────■───┼─────────■───┼────────┤ X ├─■──────────■───┼───────»
     └─┬─┘          ┌─┴─┐ │P(π/4) ┌─┴─┐ │P(-π/4) └─┬─┘          ┌─┴─┐ │P(π/4) »
q_2: ──■────────────┤ X ├─■───────┤ X ├─■──────────■────────────┤ X ├─■───────»
                    └───┘         └───┘                         └───┘         »
«
«q_0: ──■───■────────
«       │   │
«q_1: ──■───┼────────
«     ┌─┴─┐ │P(-π/4)
«q_2: ┤ X ├─■────────
«     └───┘

The workaround uses QuantumCircuit.append not QuantumCircuit.compose - append is in place as well.  The difference is that append adds things as a single instruction, whereas compose "inlines" the given circuit into the existing one.  It so happens that QuantumCircuit.control does some pretty weird stuff internally, so you probably wouldn't spot the difference in this case, but for most things, it's important for abstract optimisations in the transpiler (akin to prematurely inlining a function in a classical language).
I don't believe there's anything wrong with the QPY format itself, but I am suspicious that the recursive handling of custom gates (of which this is an example) might be skipping one of the necessary circuits when there's several compound custom gates in succession.

Oh I didn't notice the compose to append switch sorry ! Indeed that's a good fix !

Interestingly, as of 0.25.0, QPY now throws an error during the re-load, which might suggest where the underlying bug was:
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-1-2524f28d100e> in <module>
     15     dump(qc, fd)
     16     fd.seek(0)
---> 17     new_qc = load(fd)[0]
     18

~/code/qiskit/terra/qiskit/qpy/interface.py in load(file_obj, metadata_deserializer)
    267     for _ in range(data.num_programs):
    268         programs.append(
--> 269             loader(
    270                 file_obj,
    271                 data.qpy_version,

~/code/qiskit/terra/qiskit/qpy/binary_io/circuits.py in read_circuit(file_obj, version, metadata_deserializer)
   1087     custom_operations = _read_custom_operations(file_obj, version, vectors)
   1088     for _instruction in range(num_instructions):
-> 1089         _read_instruction(file_obj, circ, out_registers, custom_operations, version, vectors)
   1090
   1091     # Read calibrations

~/code/qiskit/terra/qiskit/qpy/binary_io/circuits.py in _read_instruction(file_obj, circuit, registers, custom_operations, version, vectors)
    176     else:
    177         instruction = formats.CIRCUIT_INSTRUCTION_V2._make(
--> 178             struct.unpack(
    179                 formats.CIRCUIT_INSTRUCTION_V2_PACK,
    180                 file_obj.read(formats.CIRCUIT_INSTRUCTION_V2_SIZE),

error: unpack requires a buffer of 33 bytes

I've got it: it's a bug in outputting the custom operations for a circuit that contains more than one; the breadth-first search through the new operations wasn't actually going breadth-first, it was only going through the new custom operations that appear from the last gate in the outer circuit because it was overwriting gates from before.
It's not actually anything to do with compose or control, it's solely about having multiple custom definitions that contain custom definitions of their own.  I'll provide the fix a little later today.


Set hostname to empty string if it's null so extractHostFromUrl returns something

Reviewer: @andrey-p

Description:
For whatever reason it appears that tldjs.parse(url) in extractHostFromUrl is returning an urlObj without a hostname. I would assume that this is due to tldjs choking on some kind of url, but I haven't been able to reproduce it so I'm not 100% sure. I just saw this error cropping up in the new chrome extension error page:

This just makes sure that hostname is defined so no errors are thrown.

I wonder whether we can go one step further and drop tldjs for this specific method.
The problem with tldjs is that it's extremely opinionated. If it encounters a URL that is not valid according to some specific RFC, it bails. For example, I've seen this bit cause problems:

We need it for certain cases that need something smarter, like getting the domain name without subdomains (bar.co.uk from foo.bar.co.uk). However, in this case we need the whole hostname so tldjs can be replaced with something like https://www.npmjs.com/package/url-parse.
What do you think?

Thanks for doing this BTW. We definitely need to make this stuff a less error prone.

Merging this, it's fixing errors elsewhere. We can look at the tldjs thing at a later date. :+1:

@andrey-p I realized I never followed up on your above comment. I've run into a slew of other issues stemming from tldjs as well (private TLDs are a big one). I found a fork of tldjs[1] that not only deals with the issue you mention above, but also allows us to specify that we don't want to include private TLDs in our url parsing. I think we should switch to it posthaste.
[1] https://github.com/remusao/tldts


Perform Kernal 4.8.0-beta release

Description
Need to perform Kernal 4.8.0-beta release
Affected Component
APIM
Version
No response
Related Issues
No response
Suggested Labels
No response

Kernel 4.8.0-beta release was done. Artifacts can be found at [1].
[1]. https://maven.wso2.org/nexus/content/groups/wso2-public/org/wso2/carbon/carbon-kernel/4.8.0-beta/
[2]. https://github.com/wso2/carbon-kernel/releases/tag/v4.8.0-beta


Suggestion: add config options for default xss exceptions

It would be nice if you could edit the default exceptions for the xss protection. I typically use the term 'actions' not 'action' and also would like to add a few other fields such as 'status' (usually an integer in the db).
You may also want to add the option to disable the xss protection. It's always better to have the safe way as default and be forced to disable it than to have the unsafe way default and forget to implement it, but there are those who may want to do their own escaping.
If you could throw those in it would be much appreciated. :)

Win 7 / Xubuntu VM
PHP 7.0
Laravel 5.4 (Recently upgraded from 5.3)
Laravel-Datatables 7.4

Good points. Will try to implement this when I had the chance. Thanks for the great idea.

@DMeganoski PR added as suggested. Can you please check if it's acceptable on your side? Feedbacks/suggestions appreciated. Thanks!

Awesome, thanks. :smile: Close. Just one issue. You need to use a recursive merge here otherwise, the config values for each of the excess, escape, raw, etc. columns will be overridden, not merged, if defined in the config.
Be wary though, I have had issues with array_merge_recursive() when the array contains numeric keys. I don't see this being a problem here, just a heads-up.
Other than that, seems to be perfect. I was actually going to suggest adding options for those other arrays, I don't need them at the moment but could be useful in the future. Much appreciated.

Have updated the PR and use recursive. Thanks for the feedback! Released on v7.8.0 🚀


Fix parameter shader node not declared when only connected to a VaryingSetter

Fixes #75735
But the code looks like intentional, this commit might break things. CC @Chaosus

Thanks!


How to confine handle event change to a particular cell?

HI @0x1bitcrack3r sorry for lately reply, could you please describe your requirement or question for details? I can't get you. thanks


digeq-24-view-warnings-and-errors application working but blocking su…

What
View Warnings and errors
How to test

Checkout branch and run in debug mode (questionnaire/1?debug=true)
Leave the first field empty and check an error message is presented and the form is not submitted
Populate the first field with 4 or more characters and check a warning is presented and the form is not submitted

Who can test
Anyone but @LJBabbage

Looks good, but we also need to support 'lessthan', 'greaterthan', 'equal' and 'notequal' before we can merge this. Also need to change 'max' to 'maxlength'

Also needs rebasing to master now :-1:

:+1: All good


ImList for Finals

Can ImList be used during finals? or can we only use List since ImList may not be provided.

usually if it is required, they will provide, otherwise you probably need to use 1 or to static methods from List such as List::of for the conversion to something else such as List.of(1,2,3).stream()
if really needed, ArrayList is available, and functions exactly like an ImList without the immutability

i think prof will provide ImList api, you can refer to mock paper on examplify. since we dont need to compile anything, its ok even if the qn doesnt provide implementation of ImList


Add Hostname / Nameserver Options to devices.

Currently we are putting the hostname / nameservers in comments.  Be nice to have it as option when editing a device.
As well as be able to assign Nameservers to IP addresses

Why wouldn't you use the name field for the device's name?
Miscellaneous attributes like name servers would be covered by #1349. Alternatively, you could create custom fields.

Devices often change hostname names,  so we name the device based off an internal naming system.
As far as the nameservers, we provide nameservers to our clients using our whitelabel tld, so it would be nice to be able to track them like we do IPs as they get recycled sometimes.

Yeah, sounds like custom fields are the way to go in your case.

In our case, hostnames are related to an IP, not to a device. A single device can have multiple IP which can have multiple hostnames.
It would be nice to have those hostnames showed on the Device page, based on the list of IP associated to this device.

@thefreakquency You're describing DNS management (#166). Devices each typically have exactly one hostname defined locally (e.g. hostname on Linux).


Tutorials use an abbreviated category in frontmatter

I didn't realize this until this comment but there is a manually managed relationship between the category values in the frontmatter of the tutorials and the rendered names of the categories (categoryNames in content/tutorials/config maps them). This introduces issues with translating those category names. I am curious if there are alternative options but I am not sure what the original intention was. @davepagurek what do you think?

closing after discussion here


home_work5

Hey, @Lytvynenko18 , could you please assign me to your pr, whenever they are ready for review? I would speed up the review process.


'netsh' command does not work for WPA2-Enterprise

Hi It's a great project!:two_hearts:
i found the Key Content(password ) is not displayed  when the types of networks is WPA2-Enterprise

e.g

Security settings
-----------------
    Authentication         : WPA2-Enterprise
    Cipher                 : CCMP
    Authentication         : WPA2-Enterprise
    Cipher                 : GCMP
    Security key           : Absent
    802.1X                 : Enabled
    EAP type               : Microsoft: Protected EAP (PEAP)
    802.1X auth credential : User credential
    Credentials configured : No
    Cache user information : Yes

Is there a way to solve this situation?
Thx~~~~

@xiangmingzhe0928 The password cannot be found if you are on an enterprise network. I tried it on my enterprise network and failed to retrieve the Wifi password. If I remember correctly, it is because the password is encrypted.


Add HyperlinkedImage + OrderedList components

Added 2 more components to the library which appear to be missing (???)
HyperlinkedImage

    @Test
    public void withTooltip() {
        String text = new BoldText("Example alt text").toString();
        String imageURL = "https://i.imgur.com/rP1uBWg.png";
        String redirectTo = "https://github.com/Steppschuh/Java-Markdown-Generator";
        String tooltip = "Java Markdown Generator is very useful!";
        HyperlinkedImage image = new HyperlinkedImage(text, imageURL, redirectTo, tooltip);
        System.out.println(image.toString());
    }

Would result in the following:
[![**Example alt text**](https://i.imgur.com/rP1uBWg.png "Java Markdown Generator is very useful!")](https://github.com/Steppschuh/Java-Markdown-Generator)

Which renders as (hover over):

Without tooltip:
    @Test
    public void withoutTooltip() {
        String text = new BoldText("Example alt text").toString();
        String imageURL = "https://i.imgur.com/rP1uBWg.png";
        String redirectTo = "https://github.com/Steppschuh/Java-Markdown-Generator";
        HyperlinkedImage image = new HyperlinkedImage(text, imageURL, redirectTo, null);
        System.out.println(image.toString());
    }

Would result in the following:
[![**Example alt text**](https://i.imgur.com/rP1uBWg.png)](https://github.com/Steppschuh/Java-Markdown-Generator)

Which renders as (hover over):

*Can have hyperlink as null too, with or without a tooltip*
OrderedList
    @Test
    public void renderOrderedList() {
        OrderedList list = new OrderedList();
        list.add(new OrderedListItem("An element")).add(new OrderedListItem("Another element through chaining"));
        System.out.println(list);
    }

Would create:
1. An element
2. Another element through chaining

Which results in:

An element
Another element through chaining

With builder:
    @Test
    public void orderedListWithBuilder() {
        OrderedList list = new OrderedListBuilder()
                .append("A")
                .append("B")
                .build();
        System.out.println(list);
    }

Would create:
1. A
2. B

Which results into:

A
B

Let me know if anything should be changed.

Hey, thanks a lot for your contribution!

@Steppschuh
I need this change. Can I ask for a review?

@Steppschuh I need this change. Can I ask for a review?

This PR is old and probably wouldn't merge correctly. I suggest you create your own fork, introduce the changes as appropriate, and create another pull request.

@Revxrsal
Thanks! I'll try.


Spaces in Directories/cmd

If there is a space in the cmd key's value (such as in -i /directory/ssh keys/ to specify a key to use), it is not escaped when being sent to Terminal.app, so the connection attempt fails, but if you manually escape it you get an error parsing config error in the Shuttle menu bar menu.

I have the same issue with any escaped character.

Could you please paste your config entry that causes the problem?

Going to close this ticket for now, just to clean things up. But please open a new issue if you still have problems. Also, see #86 where I had to just escape a few additional characters to get it working.

Sounds good. I'll test and open new issue if somethings now working. Thanks a lot!


[Snyk] Security upgrade ubuntu from trusty to trusty-20190515

This PR was automatically created by Snyk using the credentials of a real user.Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Changes included in this PR

vendor/golang.org/x/net/http2/Dockerfile

We recommend upgrading to ubuntu:trusty-20190515, as this image has only 337 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Some of the most important vulnerabilities in your base image include:

Severity
Priority Score / 1000
Issue
Exploit Maturity

347
Exposure of Resource to Wrong Sphere SNYK-UBUNTU1404-EXPAT-2403847
No Known Exploit

345
Improper Encoding or Escaping of Output SNYK-UBUNTU1404-EXPAT-2404052
No Known Exploit

391
Integer Overflow or Wraparound SNYK-UBUNTU1404-SQLITE3-450866
No Known Exploit

695
Off-by-one Error SNYK-UBUNTU1404-SUDO-1065770
Mature

362
Improper Handling of Exceptional Conditions SNYK-UBUNTU1404-SUDO-473059
Mature

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

vendor directory, now ignored. not code dependency for us


Add connectivity instructions or example

Description
We are looking into this library, but feel there is a missing documentation section around providing credentials and connectivity information to the testing framework. I assume it has something to do with the environment file, but I would benefit from a concrete example.
As a user who is new to behave and your new soarsdk, I believe this information would help kickstart our adoption.

In environment.py
Add this
 import requests
    from urllib3.exceptions import InsecureRequestWarning
    from urllib3 import disable_warnings

    disable_warnings(InsecureRequestWarning)
    s=requests.Session()
    s.auth=("soar_local_admin","password")

    context.phantom = PhantomClient(soar-url, session=s, verify=False)

Ah interesting. My initial thought was that I could write the feature files, install soar-behaviors and with environment variables or configuration files be able to run the tests. I have tried a variation of what @ic3guy has recommended, but exchanged the hardcoded creds for environment variables and added an option for API token instead of a local account.

Your implementation definitely sounds more production worthy.


Substitute openbeans for java.beans to support Android

Created for this issue: https://github.com/tcheeric/nostr-java/issues/118
It builds for me and solves the problem when using the library in my Android app. More details in the issue page.

Please do not hard-code dependencies' versions, use a property instead, e.g. ${openbeans.version}
The property should be defined in the main project's pom.
Thanks.


Claiming namespace subtixx

Package information

Package link:https://registry.platformio.org/libraries/subtixx/FauxmoESP

Description
It is literally my github username 😆

Have you tried to login to the PlatformIO Account using Github as a provider?

You mean in PlatformIO home? I am logged in using GitHub there if you mean that

You have automatically owned all packages. Please check it via https://docs.platformio.org/en/latest/core/userguide/access/cmd_list.html


Cut Falco 0.32.1 Release

Type
This is a patch release.
Date
Targeting the week starting July 4th, 2022
Release Steps
Read the release process for a detailed explanation.
Release Blocking PRs
This list will be updated as more issues are addressed.

https://github.com/falcosecurity/libs/pull/404
https://github.com/falcosecurity/libs/pull/393
https://github.com/falcosecurity/libs/pull/385
https://github.com/falcosecurity/test-infra/pull/734
https://github.com/falcosecurity/falco/pull/2069
https://github.com/falcosecurity/falco/pull/2078
https://github.com/falcosecurity/libs/pull/420
https://github.com/falcosecurity/libs/pull/421
https://github.com/falcosecurity/libs/pull/416
https://github.com/falcosecurity/libs/pull/425
https://github.com/falcosecurity/libs/pull/418
https://github.com/falcosecurity/libs/pull/429
https://github.com/falcosecurity/libs/pull/430

Action Items

[ ] Pre-Release

[ ] Prebuilt drivers publish (including ARM)
[x] Update milestones for closed issues to 0.32.1
[ ] Changelog
[x] Update and tag plugins

[ ] Release

[ ] Tag
[ ] GitHub Release

[ ] Website

[ ] Bump Falco version to 0.32.1
[ ] Update syscall and supported fields

[ ] Announcements
[ ] Post-Rlease

[ ] Helm Charts

Test checklist
The following items are useful for anyone that wishes to test this release when RC's are out depending on the environment they have available. Help is appreciated as always.
Installation types

Kubernetes Helm Chart
Container only (without kubernetes)
eBPF probe installation
Local DEB package
Local RPM package
Local binary
Local binary (musl-static)

0.32.1 specific checklist

https://github.com/falcosecurity/libs/pull/395

See description in the PR: "When Falco is running with -A option enabled and receives events generated by io_uring_enter and io_uring_register syscalls it crashes"

Falco-driver-loader recent modifications

https://github.com/falcosecurity/falco/pull/2066

/assign @LucaGuerra
/kind documentation

Thanks @LucaGuerra! I created a 0.32.1 milestone and moved there all currently closed PRs.

Hey @LucaGuerra
Thank you! Could you also add :point_down: please?
https://github.com/falcosecurity/falco/pull/2069

Hey @LucaGuerra
could you please add PR https://github.com/falcosecurity/libs/pull/418 ?

@LucaGuerra PTAL at these two and see if they could be useful. I think they be good nice-to-haves:

https://github.com/falcosecurity/falco/pull/2090
https://github.com/falcosecurity/falco/pull/2093

Another nice-to-have: https://github.com/falcosecurity/falco/pull/2094

@LucaGuerra can you add #2099 + #2101 as nice-to-have, and #2100 as mandatory (to fix integration tests :/ )

That libs version change in this patch bump between 0.32.0 and 0.32.1 was quite unexpected. It's not a problem, but I will call it a strange choice.
https://git.osso.nl/pub/docker/falco-ebpf-build/-/commit/76ca2dfff8d2c4835094a346768533cde1172629

Hey @LucaGuerra
The Falco Helm Chart 2.0 is out :point_down:
https://github.com/falcosecurity/charts/releases/tag/falco-2.0.0
PS I think we can close this issue now :)


Add jqiu to OWNERS

Thanks a lot for all your contributions, @qiujian16!

Thanks a lot for all your contributions, @qiujian16!

Seconded. Keep up the good work and keep the code quality high.
/lgtm
/approve

/retest

/retest


Gaussian heatmap ?

In paper, They create Ground Truth Label use Gaussian heatmap by other application. Can you show me algorithm create Gaussian heatmap? Thanks

You can use a standard normal distribution to calculate the probability associated with any pixel as a function of pixel's distance from the center.
import cv2
import numpy as np
from math import exp

# Probability as a function of distance from the center derived
# from a gaussian distribution with mean = 0 and stdv = 1
scaledGaussian = lambda x : exp(-(1/2)*(x**2))

imgSize = 512
isotropicGrayscaleImage = np.zeros((imgSize,imgSize),np.uint8)

for i in range(imgSize):
  for j in range(imgSize):

    # find euclidian distance from center of image (imgSize/2,25imgSize/25)
    # and scale it to range of 0 to 2.5 as scaled Gaussian
    # returns highest probability for x=0 and approximately
    # zero probability for x > 2.5

    distanceFromCenter = np.linalg.norm(np.array([i-imgSize/2,j-imgSize/2]))
    distanceFromCenter = 2.5*distanceFromCenter/(imgSize/2)
    scaledGaussianProb = scaledGaussian(distanceFromCenter)
    isotropicGrayscaleImage[i,j] = np.clip(scaledGaussianProb*255,0,255)

# Convert Grayscale to HeatMap Using Opencv
isotropicGaussianHeatmapImage = cv2.applyColorMap(isotropicGrayscaleImage,
                                                  cv2.COLORMAP_JET)

You can find more intuitive implementation here  IsotropiceGaussianMap Implementation using python

First, Thanks show for I GaussisionHeatmap. I use it for character  OK

But I use with link word (affinity box) to create Heatmap, it doesn't work, Becase I don't know end of word . Example

Can you show your Solution? Thanks

A working example of creating the Gaussian heat map with perspective transform(Only for character bbox not affinity bbox)
`import numpy as np
import cv2
import matplotlib.pyplot as plt
sigma = 10
spread = 3
extent = int(spread * sigma)
gaussian_heatmap = np.zeros([2 * extent, 2 * extent], dtype=np.float32)
for i in range(2 * extent):
for j in range(2 * extent):
gaussian_heatmap[i, j] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
-1 / 2 * ((i - spread * sigma - 0.5) ** 2 + (j - spread * sigma - 0.5) ** 2) / (sigma ** 2))
gaussian_heatmap = (gaussian_heatmap / np.max(gaussian_heatmap) * 255).astype(np.uint8)
def order_points(pts):
rect = np.zeros((4, 2), dtype="float32")

s = pts.sum(axis=1)
rect[0] = pts[np.argmin(s)]
rect[2] = pts[np.argmax(s)]

diff = np.diff(pts, axis=1)
rect[1] = pts[np.argmin(diff)]
rect[3] = pts[np.argmax(diff)]

return rect

def four_point_transform(image, pts):
max_x, max_y = np.max(pts[:, 0]), np.max(pts[:, 1])

dst = np.array([
	[0, 0],
	[image.shape[1] - 1, 0],
	[image.shape[1] - 1, image.shape[0] - 1],
	[0, image.shape[0] - 1]], dtype="float32")

M = cv2.getPerspectiveTransform(dst, pts)
warped = cv2.warpPerspective(image, M, (max_x, max_y))

return warped

def add_character(image, bbox):
bbox = order_points(bbox)  # bbox has shape [4, 2]
top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
bbox -= top_left[None, :]
transformed = four_point_transform(gaussian_heatmap.copy(), bbox)
image[top_left[1]:top_left[1]+transformed.shape[0], top_left[0]:top_left[0]+transformed.shape[1]] += transformed
return image

def generate_target(image_size, character_bbox):
"""
param: image_size: The original image dimensions as [channel, height, width]
param: character_bbox: The character bbox(numpy.float array) corresponding to that image having shape [2, 4, num_characters]
"""

character_bbox = character_bbox.transpose(2, 1, 0)

channel, height, width = image_size

target = np.zeros([height, width], dtype=np.uint8)

for i in range(character_bbox.shape[0]):

	target = add_character(target, character_bbox[i])

return target/255

if name == "main":
"""
A small dummy example
"""
character_bbox = np.array([[10, 10], [50, 50], [12, 38], [38, 12]]).transpose(1, 0).reshape([2, 4, 1])
image_size = [3, 100, 100]

plt.imsave('target.png', generate_target(image_size, character_bbox))`

Reference Code - https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
Do point me out if there is a bug, I will try my best to address it.

A working example of creating the Gaussian heat map with perspective transform(Only for character bbox not affinity bbox)
import numpy as np
import cv2
import matplotlib.pyplot as plt

sigma = 10
spread = 3
extent = int(spread * sigma)
gaussian_heatmap = np.zeros([2 * extent, 2 * extent], dtype=np.float32)

for i in range(2 * extent):
	for j in range(2 * extent):
		gaussian_heatmap[i, j] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
			-1 / 2 * ((i - spread * sigma - 0.5) ** 2 + (j - spread * sigma - 0.5) ** 2) / (sigma ** 2))

gaussian_heatmap = (gaussian_heatmap / np.max(gaussian_heatmap) * 255).astype(np.uint8)

def order_points(pts):
	rect = np.zeros((4, 2), dtype="float32")

	s = pts.sum(axis=1)
	rect[0] = pts[np.argmin(s)]
	rect[2] = pts[np.argmax(s)]

	diff = np.diff(pts, axis=1)
	rect[1] = pts[np.argmin(diff)]
	rect[3] = pts[np.argmax(diff)]

	return rect

def four_point_transform(image, pts):
	max_x, max_y = np.max(pts[:, 0]), np.max(pts[:, 1])

	dst = np.array([
		[0, 0],
		[image.shape[1] - 1, 0],
		[image.shape[1] - 1, image.shape[0] - 1],
		[0, image.shape[0] - 1]], dtype="float32")

	M = cv2.getPerspectiveTransform(dst, pts)
	warped = cv2.warpPerspective(image, M, (max_x, max_y))

	return warped

def add_character(image, bbox):
	bbox = order_points(bbox)  # bbox has shape [4, 2]
	top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
	bbox -= top_left[None, :]
	transformed = four_point_transform(gaussian_heatmap.copy(), bbox)
	image[top_left[1]:top_left[1] + transformed.shape[0], top_left[0]:top_left[0] + transformed.shape[1]] += transformed
	return image

def generate_target(image_size, character_bbox):
	"""
	param: image_size: The original image dimensions as [channel, height, width]
	param: character_bbox: The character bbox(numpy.float array) corresponding to that image having shape [2, 4, num_characters]
	"""

	character_bbox = character_bbox.transpose(2, 1, 0)

	channel, height, width = image_size

	target = np.zeros([height, width], dtype=np.uint8)

	for i in range(character_bbox.shape[0]):
		target = add_character(target, character_bbox[i])

	return target / 255

if __name__ == "__main__":
	"""
	A small dummy example
	"""
	character_bbox = np.array([[10, 10], [50, 50], [12, 38], [38, 12]]).transpose(1, 0).reshape([2, 4, 1])
	image_size = [3, 100, 100]

	plt.imsave('target.png', generate_target(image_size, character_bbox))

Reference Code - https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
Do point me out if there is a bug, I will try my best to address it.

A working example of creating the Gaussian heat map with perspective transform
from torch.utils import data
import matplotlib.pyplot as plt
import numpy as np
import cv2

DEBUG = True

def four_point_transform(image, pts):

	max_x, max_y = np.max(pts[:, 0]).astype(np.int32), np.max(pts[:, 1]).astype(np.int32)

	dst = np.array([
		[0, 0],
		[image.shape[1] - 1, 0],
		[image.shape[1] - 1, image.shape[0] - 1],
		[0, image.shape[0] - 1]], dtype="float32")

	M = cv2.getPerspectiveTransform(dst, pts)
	warped = cv2.warpPerspective(image, M, (max_x, max_y))

	return warped

class DataLoader(data.Dataset):

	def __init__(self, type_):

		self.type_ = type_
		self.base_path = '<Path for Images>'
		if DEBUG:
			import os
			if not os.path.exists('cache.pkl'):
				with open('cache.pkl', 'wb') as f:
					import pickle
					from scipy.io import loadmat
					mat = loadmat('Path for gt.mat')
					pickle.dump([mat['imnames'][0][0:1000], mat['charBB'][0][0:1000], mat['txt'][0][0:1000]], f)
					print('Created the pickle file, rerun the program')
					exit(0)
			else:
				with open('cache.pkl', 'rb') as f:
					import pickle
					self.imnames, self.charBB, self.txt = pickle.load(f)
					print('Loaded DEBUG')

		else:

			from scipy.io import loadmat
			mat = loadmat('Path for gt.mat')

			total_number = mat['imnames'][0].shape[0]
			train_images = int(total_number * 0.9)

			if self.type_ == 'train':

				self.imnames = mat['imnames'][0][0:train_images]
				self.charBB = mat['charBB'][0][0:train_images]  # number of images, 2, 4, num_character

			else:

				self.imnames = mat['imnames'][0][train_images:]
				self.charBB = mat['charBB'][0][train_images:]  # number of images, 2, 4, num_character

		for no, i in enumerate(self.txt):
			all_words = []
			for j in i:
				all_words += [k for k in ' '.join(j.split('\n')).split() if k!='']
			self.txt[no] = all_words

		sigma = 10
		spread = 3
		extent = int(spread * sigma)
		self.gaussian_heatmap = np.zeros([2 * extent, 2 * extent], dtype=np.float32)

		for i in range(2 * extent):
			for j in range(2 * extent):
				self.gaussian_heatmap[i, j] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
					-1 / 2 * ((i - spread * sigma - 0.5) ** 2 + (j - spread * sigma - 0.5) ** 2) / (sigma ** 2))

		self.gaussian_heatmap = (self.gaussian_heatmap / np.max(self.gaussian_heatmap) * 255).astype(np.uint8)

	def add_character(self, image, bbox):

		top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
		bbox -= top_left[None, :]
		transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))
		image[top_left[1]:top_left[1]+transformed.shape[0], top_left[0]:top_left[0]+transformed.shape[1]] += transformed
		return image

	def generate_target(self, image_size, character_bbox):

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		for i in range(character_bbox.shape[0]):

			target = self.add_character(target, character_bbox[i])

		return target/255, np.float32(target != 0)

	def add_affinity(self, image, bbox_1, bbox_2):

		center_1, center_2 = np.mean(bbox_1, axis=0), np.mean(bbox_2, axis=0)
		tl = np.mean([bbox_1[0], bbox_1[1], center_1], axis=0)
		bl = np.mean([bbox_1[2], bbox_1[3], center_1], axis=0)
		tr = np.mean([bbox_2[0], bbox_2[1], center_2], axis=0)
		br = np.mean([bbox_2[2], bbox_2[3], center_2], axis=0)

		affinity = np.array([tl, tr, br, bl])

		return self.add_character(image, affinity)

	def generate_affinity(self, image_size, character_bbox, text):

		"""

		:param image_size: shape = [3, image_height, image_width]
		:param character_bbox: [2, 4, num_characters]
		:param text: [num_words]
		:return:
		"""

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		total_letters = 0

		for word in text:
			for char_num in range(len(word)-1):
				target = self.add_affinity(target, character_bbox[total_letters], character_bbox[total_letters+1])
				total_letters += 1
			total_letters += 1

		return target / 255, np.float32(target != 0)

	def __getitem__(self, item):

		image = plt.imread(self.base_path+'/'+self.imnames[item][0]).transpose(2, 0, 1)/255
		weight, target = self.generate_target(image.shape, self.charBB[item].copy())
		weight_affinity, target_affinity = self.generate_affinity(image.shape, self.charBB[item].copy(), self.txt[item].copy())

		return image, weight, target, weight_affinity, target_affinity

	def __len__(self):

		return len(self.imnames)

if __name__ == "__main__":

	dataloader = DataLoader('train')
	image, weight, target, weight_affinity, target_affinity = dataloader[0]

	plt.imsave('image.png', image.transpose(1, 2, 0))
	plt.imsave('target.png', target)
	plt.imsave('weight.png', weight)
	plt.imsave('weight_affinity.png', weight_affinity)
	plt.imsave('target_affinity.png', target_affinity)
	plt.imsave('together.png', np.concatenate([weight[:, :, None], weight_affinity[:, :, None], np.zeros_like(weight)[:, :, None]], axis=2))

Reference Code - https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
Do point me out if there is a bug, I will try my best to address it.

In function "add_char" has bug, it is "operands could not be broadcast together with shapes (20,0) (20,29) (20,0)" This code for test dataloader
def load_data():
dataloader = DataLoader('train')
trainloader = torch.utils.data.DataLoader(dataloader, batch_size = 1, shuffle=True, num_workers=8)
for batch_idx, (image, weight, target, weight_affinity, target_affinity) in enumerate(trainloader):
print(batch_idx, ' -- ', image.shape, '--', weight.shape, '--', weight_affinity.shape)

A working example of creating the Gaussian heat map with perspective transform
from torch.utils import data
import matplotlib.pyplot as plt
import numpy as np
import cv2

DEBUG = True

def four_point_transform(image, pts):

	max_x, max_y = np.max(pts[:, 0]).astype(np.int32), np.max(pts[:, 1]).astype(np.int32)

	dst = np.array([
		[0, 0],
		[image.shape[1] - 1, 0],
		[image.shape[1] - 1, image.shape[0] - 1],
		[0, image.shape[0] - 1]], dtype="float32")

	M = cv2.getPerspectiveTransform(dst, pts)
	warped = cv2.warpPerspective(image, M, (max_x, max_y))

	return warped

class DataLoader(data.Dataset):

	def __init__(self, type_):

		self.type_ = type_
		self.base_path = '<Path for Images>'
		if DEBUG:
			import os
			if not os.path.exists('cache.pkl'):
				with open('cache.pkl', 'wb') as f:
					import pickle
					from scipy.io import loadmat
					mat = loadmat('Path for gt.mat')
					pickle.dump([mat['imnames'][0][0:1000], mat['charBB'][0][0:1000], mat['txt'][0][0:1000]], f)
					print('Created the pickle file, rerun the program')
					exit(0)
			else:
				with open('cache.pkl', 'rb') as f:
					import pickle
					self.imnames, self.charBB, self.txt = pickle.load(f)
					print('Loaded DEBUG')

		else:

			from scipy.io import loadmat
			mat = loadmat('Path for gt.mat')

			total_number = mat['imnames'][0].shape[0]
			train_images = int(total_number * 0.9)

			if self.type_ == 'train':

				self.imnames = mat['imnames'][0][0:train_images]
				self.charBB = mat['charBB'][0][0:train_images]  # number of images, 2, 4, num_character

			else:

				self.imnames = mat['imnames'][0][train_images:]
				self.charBB = mat['charBB'][0][train_images:]  # number of images, 2, 4, num_character

		for no, i in enumerate(self.txt):
			all_words = []
			for j in i:
				all_words += [k for k in ' '.join(j.split('\n')).split() if k!='']
			self.txt[no] = all_words

		sigma = 10
		spread = 3
		extent = int(spread * sigma)
		self.gaussian_heatmap = np.zeros([2 * extent, 2 * extent], dtype=np.float32)

		for i in range(2 * extent):
			for j in range(2 * extent):
				self.gaussian_heatmap[i, j] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
					-1 / 2 * ((i - spread * sigma - 0.5) ** 2 + (j - spread * sigma - 0.5) ** 2) / (sigma ** 2))

		self.gaussian_heatmap = (self.gaussian_heatmap / np.max(self.gaussian_heatmap) * 255).astype(np.uint8)

	def add_character(self, image, bbox):

		top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
		bbox -= top_left[None, :]
		transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))
		image[top_left[1]:top_left[1]+transformed.shape[0], top_left[0]:top_left[0]+transformed.shape[1]] += transformed
		return image

	def generate_target(self, image_size, character_bbox):

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		for i in range(character_bbox.shape[0]):

			target = self.add_character(target, character_bbox[i])

		return target/255, np.float32(target != 0)

	def add_affinity(self, image, bbox_1, bbox_2):

		center_1, center_2 = np.mean(bbox_1, axis=0), np.mean(bbox_2, axis=0)
		tl = np.mean([bbox_1[0], bbox_1[1], center_1], axis=0)
		bl = np.mean([bbox_1[2], bbox_1[3], center_1], axis=0)
		tr = np.mean([bbox_2[0], bbox_2[1], center_2], axis=0)
		br = np.mean([bbox_2[2], bbox_2[3], center_2], axis=0)

		affinity = np.array([tl, tr, br, bl])

		return self.add_character(image, affinity)

	def generate_affinity(self, image_size, character_bbox, text):

		"""

		:param image_size: shape = [3, image_height, image_width]
		:param character_bbox: [2, 4, num_characters]
		:param text: [num_words]
		:return:
		"""

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		total_letters = 0

		for word in text:
			for char_num in range(len(word)-1):
				target = self.add_affinity(target, character_bbox[total_letters], character_bbox[total_letters+1])
				total_letters += 1
			total_letters += 1

		return target / 255, np.float32(target != 0)

	def __getitem__(self, item):

		image = plt.imread(self.base_path+'/'+self.imnames[item][0]).transpose(2, 0, 1)/255
		weight, target = self.generate_target(image.shape, self.charBB[item].copy())
		weight_affinity, target_affinity = self.generate_affinity(image.shape, self.charBB[item].copy(), self.txt[item].copy())

		return image, weight, target, weight_affinity, target_affinity

	def __len__(self):

		return len(self.imnames)

if __name__ == "__main__":

	dataloader = DataLoader('train')
	image, weight, target, weight_affinity, target_affinity = dataloader[0]

	plt.imsave('image.png', image.transpose(1, 2, 0))
	plt.imsave('target.png', target)
	plt.imsave('weight.png', weight)
	plt.imsave('weight_affinity.png', weight_affinity)
	plt.imsave('target_affinity.png', target_affinity)
	plt.imsave('together.png', np.concatenate([weight[:, :, None], weight_affinity[:, :, None], np.zeros_like(weight)[:, :, None]], axis=2))

Reference Code - https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
Do point me out if there is a bug, I will try my best to address it.

In function "add_char" has bug, it is "operands could not be broadcast together with shapes (20,0) (20,29) (20,0)" This code for test dataloader
def load_data():
dataloader = DataLoader('train')
trainloader = torch.utils.data.DataLoader(dataloader, batch_size = 1, shuffle=True, num_workers=8)
for batch_idx, (image, weight, target, weight_affinity, target_affinity) in enumerate(trainloader):
print(batch_idx, ' -- ', image.shape, '--', weight.shape, '--', weight_affinity.shape)

Sometimes the co-ordinates of the character boxes are outside the image dimensions. In those cases, this error is being generated, I will try to update the code to incorporate the out of image dimension case, till then you can add an if else block to discard those character bboxs which have values greater than the image dimension or less than 0.

			return image

You can add this line in the add_character function

def add_character(self, image, bbox):

		top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
		bbox -= top_left[None, :]
		transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))

		start_row = max(top_left[1], 0) - top_left[1]
		start_col = max(top_left[0], 0) - top_left[0]
		end_row = min(top_left[1]+transformed.shape[0], image.shape[0])
		end_col = min(top_left[0]+transformed.shape[1], image.shape[1])

		image[max(top_left[1], 0):end_row, max(top_left[0], 0):end_col] += transformed[start_row:end_row - top_left[1], start_col:end_col - top_left[0]]

		return image

I have made these changes to the code and am not getting the error, I hope this resolves your error too.

Thanks pro. Code work for me is:
def add_character(self, image, bbox):
    if np.any(bbox < 0) or np.any(bbox[:, 0] > image.shape[1]) or np.any(bbox[:, 1] > image.shape[0]):
        return image

    top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
    bbox -= top_left[None, :]
    transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))

    start_row = max(top_left[1], 0) - top_left[1]
    start_col = max(top_left[0], 0) - top_left[0]
    end_row = min(top_left[1] + transformed.shape[0], image.shape[0])
    end_col = min(top_left[0] + transformed.shape[1], image.shape[1])

    image[max(top_left[1], 0):end_row, max(top_left[0], 0):end_col] += transformed[start_row:end_row - top_left[1],
                                                                       start_col:end_col - top_left[0]]

    return image

Base your gaussion heatmap. I implement paper. Thanks much.

Thanks, @mayank-git-hub  for your explanation of Gaussian heatmap.
Some comment about SynthText dataset is that some trascription are incorrectly labeled in SynthText. It does not guarantee that a word box corresponds to a text trancription one-to-one. Using the code below to separate the transcrition, you will obtain the start and end points of the transcription correponding to its word box.
import re
import itertools

texts = [re.split(' \n|\n |\n| ',t.strip()) for t in texts]
texts = list(itertools.chain(*texts))
texts = [t for t in texts if len(t)>0]

Welcome @YoungminBaek , @tinhchuquang !
@YoungminBaek isn't my code for creating word start and end points the same as yours in functionality?(Though your seems a bit cleaner!)
for no, i in enumerate(self.txt):
	all_words = []
	for j in i:
		all_words += [k for k in ' '.join(j.split('\n')).split() if k!='']
	self.txt[no] = all_words

@mayank-git-hub Oh, your code already has that functionality. I missed it.
Please forget about my previous comment. :)

HI! When there are more than one character in an image,is one charactor bounding box in mat['charBB'] is xmin ymin xmax ymax, as a vector or all the character in this image xmin...ymax as a vector.
I mean one line of mat[charBB][0] is [xmin, ymin, xmax, ymax] or [xmin_char1,...,ymax_char1, xmin_char2,...,ymax_char2,...]
if   [xmin_char1,...,ymax_char1, xmin_char2,...,ymax_char2,...], how to cancatnate multiple vectors which are diffrent in their length.

I want to transfrom my dataset from format  .txt to .mat

@namedysx You want create file .mat same SynthText dataset, mat['charBB'][0] has shape is [2, 4, 6], mat['charBB'][1] is [2, 4, 10] ?

@namedysx You want create file .mat same SynthText dataset, mat['charBB'][0] has shape is [2, 4, 6], mat['charBB'][1] is [2, 4, 10] ?
thanks for ur reply.
yes, and what does each channel mean?

My way is create matrix no same shape:
import numpy as np
def create_word():
    char_bb = []
    length_word = np.random.randint(1, 4) # random length word
    for i in range(length_word):
    # add 4 point of one char
          char_bb.append([np.random.randint(1, 255), np.random.randint(1, 255)])
          char_bb.append([np.random.randint(1, 255), np.random.randint(1, 255)])
          char_bb.append([np.random.randint(1, 255), np.random.randint(1, 255)])
          char_bb.append([np.random.randint(1, 255), np.random.randint(1, 255)])
    return np.array(char_bb)

char_bbs = []

char_bbs.append(create_word())
char_bbs.append(create_word())

char_bbs = np.array(char_bbs, ndmin=2) # shape = (1 , lenght(charbbs))
for i in range(char_bbs.shape[1]):
    char_bbs[0][i] = char_bbs[0][i].reshape(-1, 4, 2).transpose(2, 1, 0)

print(char_bbs.shape)
print(char_bbs[0][0].shape, char_bbs[0][1].shape)

What's the max and min value of the output gassian map score text and score link?

I still get a error, when i add it in my code
if np.any(bbox < 0) or np.any(bbox[:, 0] > image.shape[1]) or np.any(bbox[:, 1] > image.shape[0]):
return image
ValueError: operands could not be broadcast together with shapes (117,126) (0,0) (117,126)

I want to transfrom my dataset from format .txt to .mat

Hi, Can you share your code about txt to max?Thanks a lot!

@mayank-git-hub
How appropriate is the value of spread?
I think 1 is the most similar to the picture in the paper.
what do you think..?
or
Is there a way to get the normal distribution tightly in the box?

@mayank-git-hub
How appropriate is the value of spread?
I think 1 is the most similar to the picture in the paper.
what do you think..?
or
Is there a way to get the normal distribution tightly in the box?

I thought that if the affinity and the character bbox should overlap, the spread should be large so there is less chance of the word being broken. That is why I kept the spread large. I am implementing the craft training procedure, would try out will smaller values of spread and check the results. Thanks for bringing to my mind that the spread can also be a hyper-parameter to consider.
By theory the normal distribution actually should spread infinitely, but due to floating point limits it actually gets limited. You can play with the spread and configure it to the edge values you want.

Also the code I gave above
(https://github.com/clovaai/CRAFT-pytorch/issues/3#issuecomment-505903264)
breaks down if the character quadrilateral is not valid. To check you can use -
from shapely.geometry import Polygon
Polygon(bbox.reshape([4, 2]).astype(np.int32)).is_valid:
If this is False, then no need to add that character bbox

@mayank-git-hub
How appropriate is the value of spread?
I think 1 is the most similar to the picture in the paper.
what do you think..?
or
Is there a way to get the normal distribution tightly in the box?

sigma = 10
spread = 3
extent = int(spread * sigma)
center = spread * sigma / 2
gaussian_heatmap = np.zeros([extent, extent], dtype=np.float32)

for i_ in range(extent):
    for j_ in range(extent):
	    gaussian_heatmap[i_, j_] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
		-1 / 2 * ((i_ - center - 0.5) ** 2 + (j_ - center - 0.5) ** 2) / (sigma ** 2))

gaussian_heatmap = (gaussian_heatmap / np.max(gaussian_heatmap) * 255).astype(np.uint8)

This seems to work good for me.

Hi @mayank-git-hub , this gausian heat map use to gen Region score and Affinity and we use it for training right ? But as i know the gausian heatmap just the same with every char, just have diffirent about the size of annotation for each char, so why we should use it for traning ?
i use this code for gen heatmap:
in this example 1 image is 1 char
import cv2
import numpy as np
from math import exp
import matplotlib.pyplot as plt
Probability as a function of distance from the center derived
from a gaussian distribution with mean = 0 and stdv = 1
scaledGaussian = lambda x : exp(-(1/2)*(x**2))
image = cv2.imread('./test_imgs/3.jpg')
h,w,c = image.shape
isotropicGrayscaleImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
isotropicGrayscaleImage = np.zeros((imgSize,imgSize),np.uint8)
for i in range(h):
for j in range(w):
# find euclidian distance from center of image (imgSize/2,imgSize/2)
# and scale it to range of 0 to 2.5 as scaled Gaussian
# returns highest probability for x=0 and approximately
# zero probability for x > 2.5

distanceFromCenter = np.linalg.norm(np.array([i-h/2,j-w/2]))
distanceFromCenter = 2.5*distanceFromCenter/(h/2)
scaledGaussianProb = scaledGaussian(distanceFromCenter)
isotropicGrayscaleImage[i,j] = np.clip(scaledGaussianProb*255,0,255)

Convert Grayscale to HeatMap Using Opencv
isotropicGaussianHeatmapImage = cv2.applyColorMap(isotropicGrayscaleImage,
cv2.COLORMAP_JET)
plt.imshow(cv2.cvtColor(isotropicGaussianHeatmapImage,4))
plt.show()

You are assuming that the character bbox would always be horizontal which is not the case.
Also, if you create the Gaussian heatmap everytime for a new character, it would be very computationally costly.
So for generating skewed Gaussian heatmap with less computation time, you could generate a template and do perspective transformation for it as mentioned by the author in the paper.

@mayank-git-hub  I have a question, your code about four_point_transform!

@hanish3464 You could try this explanation of the perspective transform.
https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
They are trying to bring a skewed image back to horizontal rectangle using four_point_transform, while I am doing the opposite.

@hanish3464 You could try this explanation of the perspective transform.
https://www.pyimagesearch.com/2014/08/25/4-point-opencv-getperspective-transform-example/
They are trying to bring a skewed image back to horizontal rectangle using four_point_transform, while I am doing the opposite.

it mean you are using polygon annotation instead of rectangle?

Quadrilateral instead of rectangle, not polygon

@mayank-git-hub hmm the follow you export heatmap is bellow ?
Step1: read annotation get position for each char
Step2: crop image with the point from step1 and skew image to horizontal rectangle
Step3: convert img step2 -> gausian heatmap
Step4: deskew from heatmap horizontal rectangle to origin shape with Quadrilateral
it's right bro?

@mayank-git-hub hmm the follow you export heatmap is bellow ?
Step1: read annotation get position for each char
Step2: crop image with the point from step1 and skew image to horizontal rectangle
Step3: convert img step2 -> gausian heatmap
Step4: deskew from heatmap horizontal rectangle to origin shape with Quadrilateral
Step5: put back the result step4 to original image
it's right bro?

Umm, not quite, your step 2 seems to be unnecessary.

Create an isotropic square Gaussian heatmap.
Read annotation and get position of each char
Skew square Gaussian heatmap to quadilateral annotation I got from step 2
Add the output of step 3 to the target annotations

@mayank-git-hub and the isotropic gausian map gen by model trained with synthentic data right ?

First, Thanks show for I GaussisionHeatmap. I use it for character OK

But I use with link word (affinity box) to create Heatmap, it doesn't work, Becase I don't know end of word . Example

Can you show your Solution? Thanks

hi,can you show me how to use it for character?very thanks

What are the arguments to create the Gaussian heat map, width, height of heat map, the mean or the standard deviation of the distribution?

@mayank-git-hub  thanks so much for the code. Since there are multiple edits to the original code you have posted, I wanted to organize them so that others can take a quick look without having to read through the entire conversation. Please correct me if I am wrong.

1. Initial draft
from torch.utils import data
import matplotlib.pyplot as plt
import numpy as np
import cv2
from shapely.geometry import Polygon

DEBUG = True

def four_point_transform(image, pts):

	max_x, max_y = np.max(pts[:, 0]).astype(np.int32), np.max(pts[:, 1]).astype(np.int32)

	dst = np.array([
		[0, 0],
		[image.shape[1] - 1, 0],
		[image.shape[1] - 1, image.shape[0] - 1],
		[0, image.shape[0] - 1]], dtype="float32")

	M = cv2.getPerspectiveTransform(dst, pts)
	warped = cv2.warpPerspective(image, M, (max_x, max_y))

	return warped

class DataLoader(data.Dataset):

	def __init__(self, type_):

		self.type_ = type_
		self.base_path = '<Path for Images>'
		if DEBUG:
			import os
			if not os.path.exists('cache.pkl'):
				with open('cache.pkl', 'wb') as f:
					import pickle
					from scipy.io import loadmat
					mat = loadmat('Path for gt.mat')
					pickle.dump([mat['imnames'][0][0:1000], mat['charBB'][0][0:1000], mat['txt'][0][0:1000]], f)
					print('Created the pickle file, rerun the program')
					exit(0)
			else:
				with open('cache.pkl', 'rb') as f:
					import pickle
					self.imnames, self.charBB, self.txt = pickle.load(f)
					print('Loaded DEBUG')

		else:

			from scipy.io import loadmat
			mat = loadmat('Path for gt.mat')

			total_number = mat['imnames'][0].shape[0]
			train_images = int(total_number * 0.9)

			if self.type_ == 'train':

				self.imnames = mat['imnames'][0][0:train_images]
				self.charBB = mat['charBB'][0][0:train_images]  # number of images, 2, 4, num_character

			else:

				self.imnames = mat['imnames'][0][train_images:]
				self.charBB = mat['charBB'][0][train_images:]  # number of images, 2, 4, num_character

		for no, i in enumerate(self.txt):
			all_words = []
			for j in i:
				all_words += [k for k in ' '.join(j.split('\n')).split() if k!='']
			self.txt[no] = all_words

		sigma = 10
		spread = 3
		extent = int(spread * sigma)
		self.gaussian_heatmap = np.zeros([2 * extent, 2 * extent], dtype=np.float32)

		for i in range(2 * extent):
			for j in range(2 * extent):
				self.gaussian_heatmap[i, j] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
					-1 / 2 * ((i - spread * sigma - 0.5) ** 2 + (j - spread * sigma - 0.5) ** 2) / (sigma ** 2))

		self.gaussian_heatmap = (self.gaussian_heatmap / np.max(self.gaussian_heatmap) * 255).astype(np.uint8)

	def add_character(self, image, bbox):

		if not Polygon(bbox.reshape([4, 2]).astype(np.int32)).is_valid:
			return image
		top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
		if top_left[1] > image.shape[0] or top_left[0] > image.shape[1]:
			# This means there is some bug in the character bbox
			# Will have to look into more depth to understand this
			return image
		bbox -= top_left[None, :]
		transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))

		start_row = max(top_left[1], 0) - top_left[1]
		start_col = max(top_left[0], 0) - top_left[0]
		end_row = min(top_left[1]+transformed.shape[0], image.shape[0])
		end_col = min(top_left[0]+transformed.shape[1], image.shape[1])

		image[max(top_left[1], 0):end_row, max(top_left[0], 0):end_col] += transformed[start_row:end_row - top_left[1], start_col:end_col - top_left[0]]

		return image

	def generate_target(self, image_size, character_bbox):

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		for i in range(character_bbox.shape[0]):

			target = self.add_character(target, character_bbox[i])

		return target/255, np.float32(target != 0)

	def add_affinity(self, image, bbox_1, bbox_2):

		center_1, center_2 = np.mean(bbox_1, axis=0), np.mean(bbox_2, axis=0)
		tl = np.mean([bbox_1[0], bbox_1[1], center_1], axis=0)
		bl = np.mean([bbox_1[2], bbox_1[3], center_1], axis=0)
		tr = np.mean([bbox_2[0], bbox_2[1], center_2], axis=0)
		br = np.mean([bbox_2[2], bbox_2[3], center_2], axis=0)

		affinity = np.array([tl, tr, br, bl])

		return self.add_character(image, affinity)

	def generate_affinity(self, image_size, character_bbox, text):

		"""

		:param image_size: shape = [3, image_height, image_width]
		:param character_bbox: [2, 4, num_characters]
		:param text: [num_words]
		:return:
		"""

		character_bbox = character_bbox.transpose(2, 1, 0)

		channel, height, width = image_size

		target = np.zeros([height, width], dtype=np.uint8)

		total_letters = 0

		for word in text:
			for char_num in range(len(word)-1):
				target = self.add_affinity(target, character_bbox[total_letters].copy(), character_bbox[total_letters+1].copy())
				total_letters += 1
			total_letters += 1

		return target / 255, np.float32(target != 0)

	def __getitem__(self, item):

		image = plt.imread(self.base_path+'/'+self.imnames[item][0]).transpose(2, 0, 1)/255
		weight, target = self.generate_target(image.shape, self.charBB[item].copy())
		weight_affinity, target_affinity = self.generate_affinity(image.shape, self.charBB[item].copy(), self.txt[item].copy())

		return image, weight, target, weight_affinity, target_affinity

	def __len__(self):

		return len(self.imnames)

if __name__ == "__main__":

	dataloader = DataLoader('train')
	image, weight, target, weight_affinity, target_affinity = dataloader[0]

	plt.imsave('image.png', image.transpose(1, 2, 0))
	plt.imsave('target.png', target)
	plt.imsave('weight.png', weight)
	plt.imsave('weight_affinity.png', weight_affinity)
	plt.imsave('target_affinity.png', target_affinity)
	plt.imsave('together.png', np.concatenate([weight[:, :, None], weight_affinity[:, :, None], np.zeros_like(weight)[:, :, None]], axis=2))

2. new add_character
this replaces the above add_character function:
def add_character(self, image, bbox):

		top_left = np.array([np.min(bbox[:, 0]), np.min(bbox[:, 1])]).astype(np.int32)
		if top_left[1] > image.shape[0] or top_left[0] > image.shape[1]:
			# This means there is some bug in the character bbox
			# Will have to look into more depth to understand this
			return image
		bbox -= top_left[None, :]
		transformed = four_point_transform(self.gaussian_heatmap.copy(), bbox.astype(np.float32))

		start_row = max(top_left[1], 0) - top_left[1]
		start_col = max(top_left[0], 0) - top_left[0]
		end_row = min(top_left[1]+transformed.shape[0], image.shape[0])
		end_col = min(top_left[0]+transformed.shape[1], image.shape[1])

		image[max(top_left[1], 0):end_row, max(top_left[0], 0):end_col] += transformed[start_row:end_row - top_left[1], start_col:end_col - top_left[0]]

		return image

3. add checks for valid bounding boxes
This checks if bounding boxes are valid.
if np.any(bbox < 0) or np.any(bbox[:, 0] > image.shape[1]) or np.any(bbox[:, 1] > image.shape[0]):
			return image

4. new hyperparameters
this replaces the latter part of __init__ of class DataLoader:
sigma = 10
spread = 3
extent = int(spread * sigma)
center = spread * sigma / 2
gaussian_heatmap = np.zeros([extent, extent], dtype=np.float32)

for i_ in range(extent):
    for j_ in range(extent):
	    gaussian_heatmap[i_, j_] = 1 / 2 / np.pi / (sigma ** 2) * np.exp(
		-1 / 2 * ((i_ - center - 0.5) ** 2 + (j_ - center - 0.5) ** 2) / (sigma ** 2))

gaussian_heatmap = (gaussian_heatmap / np.max(gaussian_heatmap) * 255).astype(np.uint8)

@mayank-git-hub

Did you also implement code for creating affinity boxes?

Did you implement the training code?

If so, I would really appreciate if you could share!

import numpy as np
import cv2

def make_gaussian(polygon, max_xy, k=6):
    W, H = max_xy
    polygon = np.array(polygon, dtype="float32")

    # Bounding rectangle coords
    x1, y1, w, h = cv2.boundingRect(polygon[:, None, :])
    x2 = x1 + w
    y2 = y1 + h

    # Make gaussian
    cx = (x1 + x2) / 2
    cy = (y1 + y2) / 2
    sx = w / k
    sy = h / k
    x1, x2 = np.clip((x1, x2), 0, W)
    y1, y2 = np.clip((y1, y2), 0, H)
    xs = (np.arange(x1, x2)[None, :] - cx) / sx
    ys = (np.arange(y1, y2)[:, None] - cy) / sy
    gauss = np.exp(-(np.square(xs) + np.square(ys)) / 2)

    # Warp the gaussian
    w = x2 - x1
    h = y2 - y1
    if w <= 0 or h <= 0:
        return None, (None, None, None, None), (None, None)
    src = np.array([[0, 0], [0, h - 1], [w - 1, h - 1], [w - 1, 0]], dtype="float32")
    dst = polygon - np.array([x1, y1], dtype="float32").reshape(1, 1, 2)
    transform = cv2.getPerspectiveTransform(src, dst)
    gauss = cv2.warpPerspective(gauss, transform, (x2 - x1, y2 - y1))

    # True center, for regression
    # Also, rescale the gaussian
    cy, cx = np.unravel_index(np.argmax(gauss), gauss.shape)
    gauss = gauss / gauss[cy, cx]
    cx = cx + x1
    cy = cy + y1

    return gauss, (x1, y1, x2, y2), (cx, cy)

from matplotlib import pyplot as plt
W = H = 512
canvas = np.zeros((W, H), 'float32')
polygon = np.array([(109, 493), (310, 152), (148, 83), (50, 436)], 'int')
gauss, (x1, y1, x2, y2), (cy, cx) = make_gaussian(polygon, (W, H), k = 4)
canvas[y1:y2, x1:x2] = np.fmax(canvas[y1:y2, x1:x2], gauss)
cv2.polylines(canvas, [polygon], isClosed=True, color=0.5, thickness=2)
plt.imshow(canvas, cmap='jet')
plt.colorbar()


currentPage.template never works on publish

In the page.html we have:
${currentPage.template.name}
This will always result into an error on publish, because getTemplate() returns null on publish.

${pageProperties.cq:template}

should do... although you get back the path and not the name

To me the getTemplate() can better be deprecated.
cq:template indeed gives the full path, so that needs to be stripped somehow


⚠ ClamAV has degraded performance

In 2f307ac, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 62 ms

Resolved: ClamAV performance has improved in a60c904 after 5 minutes.


file upload example (addresses #387)

Coverage remained the same at 99.54% when pulling 1dd440e6b1f0c68bd0afb61f300ae7e2994324dd on spock:feature/file-upload-example into 3e69902d24326f0b8f7180d97a14cc8c54ede737 on timothycrosley:develop.

Looks great thanks!


Support newer terraform versions

We should add support for the latest terraform (currently 0.15.5)
We need to check if users are still depending on other versions - we might need to test against multiple versions - or require an upgrade 🤷♂

Duplicate of #267 ?

Closing in favour of #267


Add mapping justifications to sssom output

semapv:MappingChaining
semapv:UnspecifiedMatching

It should be in there...

Looked at the wrong output!


Consistent "Open Recent" list

Open Recent list shows full path to folder/file.

The More... list shows workspace/folder name and then full path

I think these 2 lists should have the same look where it shows workspace/folder name before the full path, with icons to easier distinguish between folder and workspace.

main

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Error: Inappropriate ioctl for device

My task in deploy.php:
require 'recipe/common.php';

// server(...);

task('deploy:start', function(){

	\Deployer\Task\Context::get()->getServer()->connect();

	run("cd {{deploy_path}}");

})->desc('Testing...');

Error in the console by calling: dep deploy:start -vvv production
➤ E➤ecuting task deploy:start
⤷ on [prod]

Run: cd /var/www/html/xxxxxxxxx.com.br/web && cd /var/www/html/xxxxxxxxx.com.br/web
> reset: standard error: Inappropriate ioctl for device
>
>
⤶ done on [prod]
✔ Ok

Including deploys applications laravel, etc.
All commands run(); return the same error.

Try to remove this line:
 \Deployer\Task\Context::get()->getServer()->connect();

I had tried this, but the error insists on continuing.
I believe they are the permissions of my server , not a problem deployer.
Anyway thanks for your help, will continue trying to solve.

Feel free to ask any questions.


Refine behavior of oauth required/optional fields.

After approving the oauth credential fronted PR, I realized that the behavior wasn't quite right.
  This applies if the oauth credential field is marked as optional or required.  Also, I made sure the 'required' tag shows up on these fields when the refresh token is filled in, because otherwise its not clear why you can't submit the form.
expires in / client secret never are required, because they may not be necessary to make the refresh token work.

Codecov Report

Merging #75 (c5fa4b8) into main (9122bd9) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head c5fa4b8 differs from pull request most recent head 6d73628. Consider uploading reports for the commit 6d73628 to get more accurate results

@@           Coverage Diff           @@
##             main      #75   +/-   ##
=======================================
  Coverage   94.21%   94.21%
=======================================
  Files          99       99
  Lines        2368     2368
=======================================
  Hits         2231     2231
  Misses        137      137

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9122bd9...6d73628. Read the comment docs.


[SPARK] Misleading error message for missing --proxy-user value

Fix misleading error message when argument is expected.
What changes were proposed in this pull request?
Change message to be accurate.
How was this patch tested?
Messaging change, was tested manually.

Can one of the admins verify this patch?

Not aware ATM, this one handles missing value for args expecting one.

Test build #3946 has started for PR 19457 at commit bc6d92e.

Test build #3946 has finished for PR 19457 at commit bc6d92e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged to master


Added x.madefor.cc to dns/domains.py

Lua-Cratestry is a module registry lua modules. Modules are uploaded to an S3 bucket for immutability. Modules are published when a git tag is pushed (via webhook).

It's still a WIP

Why "x"?

"x" is a convention used by other module registries:
golang.org/x
deno.land/x
It's short and simple to write. The plan is for modules to be available via import link:
local my_module = depend "https://x.madefor.cc/my_module/mod.lua";

Right, that sort of makes sense, thanks for explaining!
I think given madefor.cc hosts multiple people (rather than Go/Deno's websites, which are entirely owned by the respective org), I'd rather this just used the project name (or something close to it, crates or cratestry would be fine). I think I'm a little uncomfortable giving priority to one project over another.

I don't think it's considered given priority when nobody else has asked for it as far as I can see, not in my eyes anyways :D Anyway, I'm not going to argue with that. I might open a new PR later. Thx!

I don't think it's considered given priority when nobody else has asked for it

I think that's partly because nobody else has gone "wait, I could do something which isn't my project name"?

So.. What you're saying is I should rename my project to .. "x"? 😜 (just kidding)


saving gives TypeError

when i try to save i get this

these are the mods i have

disabling any of the mods fixed it. your guess is as good as mine


Losing redis connection

Sometimes, fortunately only rarely with the LLM experiment, we get the error below. We need to debug it to plan what to do. One possibility is simply to retry the connection and the failed request until it makes it. Today, if this error happens, we are likely losing data.

[flowcept][ERROR][frontier06306.frontier.olcf.ornl.gov][pid=61095][thread=140733193385728][function=_start][Connection closed by server.]
Traceback (most recent call last):
File "/lustre/orion/stf219/scratch/souzar/flowcept/flowcept/flowceptor/consumers/document_inserter.py", line 199, in _start
for message in pubsub.listen():
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1653, in listen
response = self.handle_message(self.parse_response(block=True))
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1531, in parse_response
response = self._execute(conn, try_read)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1507, in _execute
return conn.retry.call_with_retry(
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/retry.py", line 49, in call_with_retry
fail(error)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1509, in
lambda error: self._disconnect_raise_connect(conn, error),
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1496, in _disconnect_raise_connect
raise error
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
return do()
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1508, in
lambda: command(*args, **kwargs),
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/client.py", line 1529, in try_read
return conn.read_response()
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 848, in read_response
response = self._parser.read_response(disable_decoding=disable_decoding)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 335, in read_response
result = self._read_response(disable_decoding=disable_decoding)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 383, in _read_response
response = [
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 384, in
self._read_response(disable_decoding=disable_decoding)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 377, in _read_response
response = self._buffer.read(length)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 230, in read
self._read_from_socket(length - self.length)
File "/lustre/orion/stf219/scratch/souzar/miniconda/envs/llm3/lib/python3.8/site-packages/redis/connection.py", line 195, in _read_from_socket
raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
redis.exceptions.ConnectionError: Connection closed by server.

I found that it is an intermittent error that happens on Frontier, likely due to network issues. Anyhow, we might need to consider handling this failure better than just missing the data.


HOWTO confirm your email address after registration

After registration, go to the  email address provider you registered with.  There is a link from 1millionwomentotech that will ask you to confirm the email address.  Click on this link.  You're done!

Hi @ilonabudapesti , please refer to PR #358


1.19 support

Really cool mod, but only on 1.18.2

I have not many spare time.But I will do it.
| |
郭岫辰
|
|
@.***
|
---- Replied Message ----
| From | @.> |
| Date | 10/23/2022 19:31 |
| To | @.> |
| Cc | @.***> |
| Subject | [Jack253-png/CAnimation] 1.19 support (Issue #1) |
Really cool mod, but only on 1.18.2
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Project documentation update

Summary
A project documentation update is required.
The currently listed projects do not follow any specific flow and do not detail the work or the efforts property.
There is a ton of paper documentations that can be scanned and attached, better write up and updates about the project. Best done alongside development hence a flow structure to be incorporated as well.
Motivation
Sharing would be nice.
Drawbacks

Time

Move flutter arsenal from labs to project

#33 is ongoing PR addressing the same


Deploy test

GH pages isn't deploying, trying with a PR.


Conflict with hardhat-ethers when running tsc

When we run tsc we get these errors:
node_modules/@nomiclabs/hardhat-ethers/dist/src/type-extensions.d.ts:6:10 - error TS2300: Duplicate identifier 'Libraries'.

6     type Libraries = LibrariesT;
           ~~~~~~~~~

  node_modules/hardhat-deploy/dist/src/type-extensions.d.ts:261:10
    261     type Libraries = {
                 ~~~~~~~~~
    'Libraries' was also declared here.

node_modules/hardhat-deploy/dist/src/type-extensions.d.ts:261:10 - error TS2300: Duplicate identifier 'Libraries'.

261     type Libraries = {
             ~~~~~~~~~

  node_modules/@nomiclabs/hardhat-ethers/dist/src/type-extensions.d.ts:6:10
    6     type Libraries = LibrariesT;
               ~~~~~~~~~
    'Libraries' was also declared here.

Found 2 errors.

Thanks for the report,
fixed in  0.7.0-beta.21
https://github.com/wighawag/hardhat-deploy/blob/master/CHANGELOG.md#070-beta21

I still see the error below when doing tsc.
node_modules/hardhat-deploy/dist/src/type-extensions.d.ts:3:47 - error TS2307: Cannot find module '../types' or its corresponding type declarations.

3 import { Address, DeploymentsExtension } from '../types';
                                                ~~~~~~~~~~

Found 1 error.

error Command failed with exit code 2.

Looks like it should be this.
- import { Address, DeploymentsExtension } from '../types';
+ import { Address, DeploymentsExtension } from './types';

Hopefully fixed in 0.7.0-beta.22


[KYUUBI #3072][DOC] Add a doc of Flink Table Store for Flink SQL engine

Why are the changes needed?

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Codecov Report

Merging #3234 (f389629) into master (e847ab3) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #3234      +/-   ##
============================================
- Coverage     51.68%   51.66%   -0.03%
- Complexity        6       13       +7
============================================
  Files           464      465       +1
  Lines         25651    25722      +71
  Branches       3551     3561      +10
============================================
+ Hits          13259    13290      +31
- Misses        11131    11157      +26
- Partials       1261     1275      +14

Impacted Files
Coverage Δ

.../kyuubi/server/mysql/constant/MySQLErrorCode.scala
13.84% <0.00%> (-6.16%)
:arrow_down:

...ache/kyuubi/server/mysql/MySQLCommandHandler.scala
76.13% <0.00%> (-3.41%)
:arrow_down:

...ache/kyuubi/server/mysql/MySQLGenericPackets.scala
76.59% <0.00%> (-2.13%)
:arrow_down:

...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala
78.00% <0.00%> (-2.00%)
:arrow_down:

...ala/org/apache/kyuubi/util/KyuubiHadoopUtils.scala
95.00% <0.00%> (-1.78%)
:arrow_down:

...ache/kyuubi/operation/KyuubiOperationManager.scala
80.82% <0.00%> (-1.37%)
:arrow_down:

...mon/src/main/scala/org/apache/kyuubi/Logging.scala
43.42% <0.00%> (-1.32%)
:arrow_down:

...yuubi/server/metadata/jdbc/JDBCMetadataStore.scala
89.27% <0.00%> (-0.70%)
:arrow_down:

...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala
82.60% <0.00%> (-0.63%)
:arrow_down:

...ubi/engine/spark/SparkTBinaryFrontendService.scala
82.27% <0.00%> (-0.62%)
:arrow_down:

... and 4 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thanks, merging to master


OSS: 2.6.1 - Getting 404 accessing the node.js setup page

Steps to reproduce:

Install OSS 2.6.1 (create ORG, users etc etc)
Go to http://localhost:8086 and authenticate
Place cursor over 'Up arrow' icon, displaying sub-menu "Sources|Buckets|Telegraf|Scapers|API Tokens"
Select "Sources"
Click JavaScript/Node.js under client Libraries

Observed: Getting 404 Page Not Found
Expected: Directed to the node.js Setting up Page
Environment info:

System info: Run uname -srm and copy the output here
Linux 4.18.0-408.el8.x86_64 x86_64

InfluxDB version: Run influxd version and copy the output here
InfluxDB v2.6.1 (git: 9dcf880fe0) build_date: 2022-12-29T15:53:07Z

Other relevant environment details: Container runtime, disk info, etc
CentOS Stream release 8
influxdb2-2.6.1-1.x86_64
influxdb2-cli-2.6.1-1.x86_64

Config:
bolt-path = "/var/lib/influxdb/influxd.bolt"
engine-path = "/var/lib/influxdb/engine"
flux-log-enabled = true
ui-disabled = false
hardening-enabled = false

ScreenShot

In addition, getting the same 404 page selecting go & python client libraries.

Thanks Rick, appreciate you raising this.

I think you can still access the page by manually entering the url like this:
http://localhost:8086/orgs/<your org id>/load-data/client-libraries/python

I think you can access the page by manually entering the url like this:
http://localhost:8086/orgs/<your org id>/load-data/client-libraries/python

You still need to generate a token using CLI though.

Thanks but getting the same result, 404.

Has someone an idea how to install the client library for python? I got the same 404 issue. Had installed influxDB in a Docker on my NAS.

For installing the client library, check this.
You need to have a token for the python client to connect to the db and access a bucket. You can use the admin token although not recommended. Proper way of doing it is

create a bucket (in the console or using bucket cli)
create a token that has read/write access to the bucket  (in the console or using auth cli)
in the client app, use the token to access the bucket

Is there a fix for this anywhere? I am having the exact same issue as OP. I want to add a node.js source to a new influxdb instance but get the 404 error. Browsing directly gets the same result.
I tried several threads to install client libraries but non have worked so far.
I did have this running in a docker container on my NAS but want to get it running separately so trying to run on a raspberry pi 3B running Raspberry Pi OS Lite (64 Bit).
I have the same setup running on another Pi (same spec) gathering data from a different source perfectly. The instance also giver the 404 error when I try to add a node.js source.
My Environment:
$ uname -srm
Linux 5.15.84-v8+ aarch64
$ influxd version
InfluxDB v2.6.1 (git: 9dcf880fe0) build_date: 2022-12-29T15:52:06Z
bolt-path = "/mnt/influxdb/influxd.bolt"
engine-path = "/mnt/influxdb/engine"
Any suggestions appreciated :)

It is being worked on, but in the meantime all the information on utilizing the javascript client can be found https://github.com/influxdata/influxdb-client-js.
innomatica's suggestions above are correct in that you will want to create a new token that is limited to just the scope that you need. The links they provided will help with that.