sfdx force:mdapi:deploy does nothing

Summary
We use the sfdx force:mdapi:deploy command to deploy to our packaging org. We have been following #1508 and trying workaround steps as well as upgrading version 7.151.1 and the result is still the same. When viewing the log file we receive this:
{"name":"sfdx","hostname":"Jamess-MacBook-Pro.local","pid":39488,"log":"SfdxProjectJson","level":40,"msg":"The config file: /Users/jameshafer/.sfdx/sfdx-project.json is not schema valid\nDue to: Validation errors:\n#/required: must have required property 'packageDirectories'","time":"2022-05-19T16:09:13.067Z","v":0}
This seems to be searching for a sfdx-project.json at a global level rather than the project level.
Steps To Reproduce:

sfdx force:source:convert -d build
sfdx force:mdapi:deploy -u PackagingOrg -d build -w 10

I am cannot provide the repository.
Expected result
The latest changes are deployed to the packaging org.
Actual result
Nothing happens, it does not attempt to deploy or provide an error.
System Information
{
"cliVersion": "sfdx-cli/7.151.1",
"architecture": "darwin-x64",
"nodeVersion": "node-v14.17.0",
"pluginVersions": [
"@oclif/plugin-autocomplete 0.3.0 (core)",
"@oclif/plugin-commands 1.3.0 (core)",
"@oclif/plugin-help 3.3.1 (core)",
"@oclif/plugin-not-found 1.2.6 (core)",
"@oclif/plugin-plugins 1.10.11 (core)",
"@oclif/plugin-update 1.5.0 (core)",
"@oclif/plugin-warn-if-update-available 1.7.3 (core)",
"@oclif/plugin-which 1.0.4 (core)",
"@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)",
"alias 2.0.0 (core)",
"apex 0.11.0 (core)",
"auth 2.0.2 (core)",
"community 1.1.4 (core)",
"config 1.4.6 (core)",
"custom-metadata 1.1.0 (core)",
"data 2.0.0 (core)",
"generator 2.0.1 (core)",
"info 2.0.0 (core)",
"limits 2.0.0 (core)",
"org 1.13.1 (core)",
"salesforce-alm 54.4.0 (core)",
"schema 2.1.0 (core)",
"sfdx-cli 7.151.1 (core)",
"signups 1.1.0 (core)",
"source 1.9.8 (core)",
"telemetry 2.0.0 (core)",
"templates 54.7.0 (core)",
"trust 1.1.0 (core)",
"user 1.7.1 (core)"
],
"osVersion": "Darwin 21.1.0"
}
Additional information
Please let me know if you need any other information as this is blocking us from releasing a new package, thank you!

There can be a sfdx-project.json set at the global level for things that should apply to all projects.  This is not commonly used, but that's also just a warning and not causing any problems.
Just to confirm, the convert happened successfully right?  I.e., the contents of the build dir looks correct?
For the deploy command, please append --dev-debug and see if there is something in the output that provides more clues.
As a workaround, you might want to try using sfdx force:source:deploy -u PackagingOrg, which should do the same thing.

Yes, the convert is successful and the build dir looks correct.
I'm not seeing anything that shows any detail to why it is failing when running the debug.
The workaround did work, however, it also has some strange behavior. In order to use that command you need to re auth to the org each time you need to deploy. If you don't you receive the following error:
The encrypted data is not properly formatted. You also get the same error when using
*** Internal Diagnostic ***
InvalidEncryptedFormatError: The encrypted data is not properly formatted.
at AuthInfoCrypto.decrypt (/usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/@salesforce/core/lib/crypto.js:112:19)
at AuthInfoCrypto.crypt (/usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/@salesforce/core/lib/authInfo.js:135:45)
at AuthInfoCrypto.decryptFields (/usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/@salesforce/core/lib/authInfo.js:124:21)
at AuthInfo.refreshFn (/usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/@salesforce/core/lib/authInfo.js:693:48)
at SessionRefreshDelegate.refresh (/usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/jsforce/lib/http-api.js:283:15)
at /usr/local/lib/node_modules/sfdx-cli/node_modules/@salesforce/source-deploy-retrieve/node_modules/jsforce/lib/http-api.js:91:23
at tryCallOne (/usr/local/lib/node_modules/sfdx-cli/node_modules/promise/lib/core.js:37:12)
at /usr/local/lib/node_modules/sfdx-cli/node_modules/promise/lib/core.js:123:15
at flush (/usr/local/lib/node_modules/sfdx-cli/node_modules/asap/raw.js:50:29)
at processTicksAndRejections (internal/process/task_queues.js:77:11)
This same error occurs when using the legacy deploy. With the workaround this is not as urgent but I would still like to help get this fixed for the future, please let me know if you need anything else.

I don't know if this is exactly the same issue but I get the same thing - running sfdx force:mdapi:deploy does absolutely nothing.
No output on stdOut or stdErr. Tried --dev-debug and there is nothing obviously wrong at all. We've used the exact same command for years.
Using the legacy command does work so we've updated all of our build tooling to use the legacy versions but we do need to get to the bottom of why this isn't working. Very concerned that a release has gone out where the main deploy command doesn't work at all and no tests have caught it...

I have the same issue, absolutely nothing in the deployment status screen as well

This appears to be the same issue as #1508. Downgrading sfdx CLI worked as a workaround for us.

I don't think it's exactly the same as no deployment is even initiated in this case

This worked for me sfdx force:mdapi:legacy:deploy

Thanks @sankeerth999 , Its helped me a lot

same for  me

This issue was fixed with CLI v7.156.1


Put mixins in their place (part 2)

Looks like we need to do this in two parts in order to preserve history. I'll merge part 1 but not release, then merge part 2 shortly after.
Part 1: move the mixin files into their own directories and adjust all paths within core
Part 2: restore the original mixin files but just have them export from the new location

FYI we get this sort of history in GH now:

Which seems reasonable - following the link shows the rest of the history.
And blames look perfect, e.g. for focus-mixin!


can not update value in  phaser object from method that is called from dom event

Hi,
the issue is when try to update an variable in the phaser object class , not from [create(),preload(),update() etc..) , no thing get happens , as if no modification is done
this post has the same problem with code , so it may help.
Forum post
More Description :
i am using a textbox to get input for the game , when the user enters a text i call a method that set the value to variable in the class that inherits from Phaser.State , but the value not get updated
phaser version :  Kore Springs 2.6.2
web framework : angular 2

Thank you for taking the time to open this issue. However, official support for Phaser 2 has now ended as we focus on preparing Phaser 3 for beta release. In a bid to keep the project tidy, and avoid confusion, we are closing all v2 related issues.
This does not mean your issue won't be looked-at. In November 2016 the Phaser CE (Community Edition) project began. The aim of Phaser CE was to allow the community to continue v2 development. Since then it has gained significant traction, with many updates and releases.
If you are still interested in having your issue investigated we strongly recommend testing the latest Phaser CE release. And only if the problem persists, opening an issue on the Phaser CE repo.


Any hints on Deep Successor Representation (DSR)

Hey Arthur,
I hope your doing fine!
I'm currently implementing a tensorflow version of the paper "Deep Successor Reinforcement Learning" for my master thesis. Somehow the learning is really unstable even on a simple 3d gridworld environment. Have you worked with that paper?

Hi @SaifAlDilaimi,
Unfortunately I have not worked with that architecture before, although I am familiar with it. When it comes to deep reinforcement learning models, many of the theoretical guarantees regarding convergence are no longer applicable. As such I am not sure what you should expect from that model on your task. One thing I would recommend is to look at other open source implementations of the same model and see if there are any significant discrepancies between what you are doing and what they did.
Sorry I can't be of more help.

Hi Arthur,
thank you for answering! Really appreciate it!
I have looked on a those few implementations that refer to it. All of them use a dqn that is solely responsible for the q-values. However, in the mentioned paper they compute the q-values by taking the dot product of the successor features and the weights of the reward vector. They also compute a target a' in the loss function which makes no sense to me.
Maybe it's too much to ask, but would you mind taking a few minutes to look at my implementation? You can find it on the following github repo:
https://github.com/SaifAlDilaimi/OpenAIGym-DSR/blob/main/dsr_example.py
Any help, no matter how small, is greatly appreciated.
PS: No worries if thats to much.. Thank you anyway!


Constructing a Bezier of two points and epsilon fails

The method PolyLine2d Bezier.bezier(double epsilon, final Point2d... points) crashes when it tries to compute the first t value to put the points in the NavigableMap. When only two points are given, the value of t is effectively computed as 0/0; resulting in NaN.
It is likely that this also goes wrong in the PolyLine3d version.
This should first be detected with a new or improved unit test and only then be fixed.

Fixed. Also improved test coverage of 3D bezier construction with epsilon (this was not tested at all).


Add nonblocking stress test and adjust blocking test.

Modify the blocking test to do repeated connections to the database. The
Nonblocking test is then based on the blocking test, but is parametrised
over Mariadb.Nonblocking.Wait and instantiated for lwt and async.
Comment
This extends the tests to cover #18.  Let me know if you want adjustments.  It could be useful to generalise it to attempt a pass-though of arbitrary parameter vectors, but I have to understand better casts done by MariaDB, and this form may be more useful at the moment for the given issue.

I'll take a look at the generalised pass-though test in the evening, in case you want it included in this PR.


feat: unblock event queue when network events are blocked

Furthermore, update some E2E tests so that they do not rely on CDP events.
Bug #644
Co-authored-by: Maksim Sadym<EMAIL_ADDRESS>Re-created from #1400.

It's just missing "continue request", but sending out for review already.


[$250] IOU - Correct digit is not getting deleted in IOU request reported by @daraksha-dk

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Request Money > type 1234
Click between 3 & 4 and hold the cursor and move to any other position (b/w 1&2 or 2&3 etc)
Now hit delete

Expected Result:
Deletion should happen from where the cursor is
Actual Result:
only the initial position is getting lost (it will delete 3)
Workaround:
unknown
Platform:

Where is this issue occurring?

Mobile Web - Android FF

Version Number: 1.2.21-4
Reproducible in staging?: y
Reproducible in production?: y
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/43996225/199030369-a5e4046c-8dfc-4468-b33f-fe70793757e3.mp4
https://user-images.githubusercontent.com/43996225/199030408-e4025207-f807-46eb-8b7b-deca7efec076.mp4
Expensify/Expensify Issue URL:
Issue reported by: @daraksha-dk
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1667200193320029
View all open jobs on GitHub

Sounds like a legit bug. Pushing this to external to get proposals for it.

Job posted:
https://www.upwork.com/ab/applicants/1587512114154676224/job-details
https://www.upwork.com/jobs/~01cdae73aeb040edd9

@kavimuru I'm trying to tackle this but I'm not able to reproduce this. I tried reproducing with the Brave and Chrome browsers. What browser are you using?

@AndreasBBS It happens only in Android - Firefox browser

I've investigated the issue a bit and here are my findings so far:

On Chrome the nativeEvent that react-native sends to onSelectionChange is of type: "selectionchange"
On Firefox the nativeEvent that react-native sends to onSelectionChange is of `type: "mouseup"
Both in Chrome and Firefox the event is triggered when the user presses somewhere in the text, so the component is able to update the selection values correctly on both browsers in this case
On Chrome when the cursor is "dragged" the event is fired every time the cursor moves a character position, so the component is able to update the selection values correctly
On Firefox when the cursor is "dragged" the event is not fired, so the component does not know it has to update the selection values. This causes the component to "think" the cursor is still in the position where the user originally pressed. It not only causes the wrong digit to be deleted (as it is mentioned here in this post) but also if you introduce a new digit it will not place the digit in the cursor position but instead it will place it in the original cursor position (before dragging). I've verified that this variant of the bug also happens.

Proposal
My proposal is that this bug needs to be solved at the react-native library level. React-native should trigger the events in the same way regardless if it's Firefox or Chrome and I defend that this should be the approach to fix instead of changing the code of this App. The App code does the correct thing and when the events are triggered correctly, as is the case in Chrome, the correct behavior is observed.

@Santhosh-Sellavel - could you review this proposal?

@AndreasBBS
Thanks for investigating!
Have you checked on the react-native issues is there any related bugs?
Do you have a solution for the bug, if so please share it so I can review it.

Bumped back to daily per our new BugZero process, internal Slack thread with deets

@Santhosh-Sellavel I've checked both the react-native and react-native-web issues and didn't find anything relating to this bug.
Because this is a browser bug I opened an issue in the react-native-web repo, you can find it here https://github.com/necolas/react-native-web/issues/2422
You can also find a codesandbox there that illustrates the bug.
I looked into the react-native-web source and I couldn't really come up with a solution for this bug. I'm hoping that dialogue in the issue I created there might lead to some better solutions. I also think that this possible needs to be solved at the Firefox engine level because the native event is react-native-web is just the events that come from the browser and Firefox does not seem to fire the events correctly in this dragging cursor case. I'll wait for some replies in the react-native-web issue I created but I suspect this might prove to be a symptom of Firefox engine...
I'd love if you could try the codebox example I left on the react-native-web issue and give me your thoughts. This seems to me to be an issue outside of the scope of this app and fixes at lower lever libraries probably will require a higher insensitive than $250.

Thanks again, I've checked out codesandbox. The issue is occurring at my end as well on the Firefox browser.
https://user-images.githubusercontent.com/85645967/200191556-f257d100-fa0d-48dd-b012-34cb6b50ccb0.mp4
Let's wait for some response from react-native-web libraries as well, but the issue seems to be at from firefox engine level.
Regarding incentives, if we have a clear path toward a solution, we could definitely discuss it.
Read this as we stated,

Please be aware that compensation for any support in solving an issue is provided entirely at Expensify’s discretion. Personal time or resources applied towards investigating a proposal will not guarantee compensation. Compensation is only guaranteed to those who propose a solution and get hired for that job.

I've found this as well and I'm gonna drop it here cause it might be useful to somebody trying to tackle this issue: https://developer.mozilla.org/en-US/docs/Web/API/HTMLInputElement/selectionchange_event
The example in the link works on Firefox for Android but it does not work in Firefox for Chrome.
It seems to me that Chrome and Firefox deal with this selection feature differently and react-native-web is defaulting to Chrome's solution...

Update: I've investigated further and I discovered that this bug is something that extends to the react package. react-native-web uses react's input onSelect for the TextInput onSelectionChange handler. I closed the issue on react-native-web and opened a new one on react's repository: https://github.com/facebook/react/issues/25643

Just clarifying @AndreasBBS - this GH needs to be on hold until the other you mentioned is fixed right?

Just clarifying @AndreasBBS - this GH needs to be on hold until the other you mentioned is fixed right?

Yes, right. I'm not getting a lot of feedback on that issue and it seems to me that issues in that repo take a while to be addressed.
I've been trying to fix it myself but react source is dense and I rarely have to work at this level, I'm enjoying the challenge though. If you know anyone that's familiar with working on the react source code that can give me some guidance I'd highly appreciate if you could connect me with.

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!
Join our slack channel, and post help anyone can offer help!

Price doubled - https://www.upwork.com/jobs/~01cdae73aeb040edd9

Issue not reproducible during KI retests. (First week)

Hey @Santhosh-Sellavel - Does this have relevant proposals? Is the PR that @AndreasBBS put up related?

Is https://github.com/Expensify/App/pull/12632 that @AndreasBBS put up related?

No Its not related!

We have some investigation, but not a complete proposal!

Great, going to double the price.

@tgolen this is one of the oldest issues in the /App repo. To help us clear out the large backlog of bugs, can you:

Decide whether any proposals currently meet our guidelines and can be approved as-is
For any that can't, please take this issue internal and treat it as one of your highest priorities
If you have any questions, don't hesitate to start a discussion in #bug-zero

Considering that this doesn't impact any of our supported platforms, I am going to close this out. There is an upstream fix in react-native that @AndreasBBS opened, and I think it's still good to pursue that, and I think closing this in the meantime is the right move.

I actually moved the upstream fix to react itself once I realized the bug is originating from that package and not react-native. You can follow the issue here: https://github.com/facebook/react/issues/25643


feat: add 1.4.1 contracts for Morph Mainnet

Add new chain
Please fill the following form:
Provide the Chain ID (Only 1 chain id per PR).

Chain_ID: 2818

Provide RPC URL for the chain (should be able to query atleast 3+ requests per second for automatic PR check).

RPC_URL: https://rpc.morphl2.io

Relevant information:

https://github.com/ethereum-lists/chains/blob/master/_data/chains/eip155-2818.json
https://github.com/safe-global/safe-singleton-factory/issues/740

Could you also please deploy the migration contracts?

@remedcu let me check, I suppose we have already deployed them :eyes:

Hey @remedcu :wave:
I apologize, I missed these few migration contracts for this PR. Thanks for so quick review!


Support for Angular 5+

This is a...

[x] feature request
[ ] bug report
[ ] usage question

Additional Notes:
Currently this project does not support Angular 5.0.
Is there any pre-release that does?
What do we need to change to implement the support?

Hi,
Just put out a pre-release of v6.6.0-1 - this just updated the peer deps to include Angular 5, initial testing seems like it works without issue, although I don't have any large apps on V5 yet to fully test - any feedback is appreciated.
There is also v7.0.0-2 - which updates the typings, and also uses the newest version of the angular compiler/etc, but produces metadata/builds that don't work with Angular 4.

v7 has now been published - if you have any issues upgrading feel free to let me know / raise an issue.
Thanks for your patience.


Device Stack Refactor

The design of the device stack is more or less entirely overhauled in favor of a more structured and "Rustic" state machine design. The design and state of the stack is more fully discussed in the device/README.md.
In addition, the embedded concerns have been removed from this project in favor of pushing them into sx12xx-rs. This will make this allows this repository to be little more streamlined.

Thank you for your contribution and for working with me to have it well aligned with my vision of the encoding API :slightly_smiling_face:


Output is not generated.

Hi,
I used the below snippet. Although it does not return any error but it does not produce any output. I think the code from line mailListener.on does not execute (Please help.!!!
var MailListener = require("mail-listener2");
	 var mailListener = new MailListener({
	   username<EMAIL_ADDRESS>	   password: "test",
	   host: "imap.gmail.com",
	   port: 993, // imap port
	   tls: true,
	   connTimeout: 10000, // Default by node-imap
	   authTimeout: 5000, // Default by node-imap,
	   debug: console.log, // Or your custom function with only one incoming argument. Default: null
	   tlsOptions: { rejectUnauthorized: false },
	   mailbox: "INBOX", // mailbox to monitor
	   searchFilter: ["UNSEEN", "FLAGGED"],
	   markSeen: true, // all fetched email willbe marked as seen and not fetched next time
	   fetchUnreadOnStart: true, // use it only if you want to get all unread email on lib start. Default is `false`
	   mailParserOptions: {streamAttachments: true}, // options to be passed to mailParser lib.
	   attachments: true, // download attachments as they are encountered to the project directory
	   attachmentOptions: { directory: "attachments/" }
	 });

	 mailListener.start();
	 mailListener.on("server:connected", function(){
	   console.log("imapConnected");
	 });

	 mailListener.on("mail:arrived", function(id){
	   console.log("new mail arrived with id:" + id);
	 });

	 mailListener.on("mail:parsed", function(mail){
	   // do something with mail object including attachments
	   console.log("emailParsed", mail.attachments);
	   // mail processing code goes here
	 });

@mchopra1004
Add an error handler to log any errors during setup
mailListener.on("error", function(err){     console.log(err);   });
If "imapConnected" is logged on a successful connection my next issue was the searchFilter property. I had to remove the "FLAGGED" filter to see requests

Same here. Removing "FLAGGED" value from searchFilter made my code finally see Gmail emails. Thank you @rphansen91!

I also had to remove "FLAGGED" from the filter to be able to see emails from an iRedMail server.

After running the above code getting Error
mime.extension is not a function

I get the same as this..
If "FLAGGED" is set, then I see debug log that messages are arrived but the on("mail".. handler does not fire.
If I remove FLAGGED, then the parsing is failing with mime.extension is not a function.


Chakra UI

merge

this cannot be merged to master. We  could create a branch for it in the examples repo? scaffold-eth-examples.


README Update

README Update

Adding installation instructions for Debian unstable/testing/buster-backports
and Ubuntu Focal
Added a badge to show the  package version in Debian Unstable

Is there any reason the badge says 1.4.1-2? I think in general, it's best to skip the badge altogether. Maybe to be consistent I should remove the snap badge.

Is there any reason the badge says 1.4.1-2? I think in general, it's best to skip the badge altogether. Maybe to be consistent I should remove the snap badge.

because the version in Debian unstable at the moment is that.

Although the new version is uploaded by @utkarsh2102, the newer version has not hit the repositories yet. Once that happens, the version will change to 2.0.0-1
Also you should keep the snap badge, better to get the info looking at the badge statuses.

Ok great thanks for the info!

I will add the badge to the readme once micro becomes generally available in apt.

#1500 has been merged.


Fix Write-Message -Level Error

This no longer exists need to replace by the Stop-Function
This also fix the issue #2754

merci! 🌻


feat: add middleware support to fastify

This PR adds support for middleware in the fastify implementation. This is heavily inspired by the express implementation so the code should look somewhat familiar :)
First-class middleware support was removed from Fastify after v3 and now relies on plugins like @fastify/express @fastify/middie etc.
If this PR is good to be merged, I can also update the docs to reflect these changes too.
One major point is that the middleware can only be used if the router is installed as a plugin (#252) because the middleware support is also plugin based and in order to get access to the plugin context, we also need to be in the plugin context.

Codecov Report

Merging #265 (cccde64) into main (2763208) will not change coverage.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@           Coverage Diff           @@
##             main     #265   +/-   ##
=======================================
  Coverage   58.95%   58.95%
=======================================
  Files          20       20
  Lines         441      441
  Branches      112      112
=======================================
  Hits          260      260
  Misses        170      170
  Partials       11       11

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

This looks great - nice and aligned with the new express middleware (bar global, assuming because you can write it outside of ts-rest?)
@Gabrola would be great to hear your thoughts also 🙌

I think Fastify hooks have replaced the middleware functionality without needing any plugins and can still be added either globally or per-route, with the added benefit of the granular hook types (onRequest, preHandler, etc.).
Also from this page there seems to be a major limitation regarding the use of middleware:

Fastify middleware does not expose the send method or other methods specific to the Fastify Reply instance. This is because Fastify wraps the incoming req and res Node instances using the Request and Reply objects internally, but this is done after the middleware phase. If you need to create middleware, you have to use the Node req and res instances. Otherwise, you can use the preHandler hook that already has the Request and Reply Fastify instances. For more information, see Hooks.

I'm not super familiar with the fastify ecosystem but some quick research indicates that popular packages like @fastify/passport depend on hooks rather than middleware, and they cannot be used interchangeably because of the different exposed objects as quoted above. So to support these packages we must add hook functionality.
Personally, I think its better to just leave behind an officially unsupported system behind while not needing to rely on extra external packages, and it seems like it's a relatively easy switch from the perspective of this code.

After reading some more on this, I think you're right @Gabrola and the approach I took doesn't really work well with Fastify anymore.
My main reason for going this way was to align with ts-rest's express API but I'm now sure that this isn't the right way.
Taking into account fastify's hook system, what do we think about an API like this:
const router = s.router(contract, {
  hooks: {
    preValidation: (request, reply) => {
      // single hook
    },
    onRequest: [
      (request, reply) => {
        // array of hooks
      }
    ]
  },
  getPosts: {
    handler: async (tx) => {
      return {
        status: 200,
        body: []
      }
    }
  }
});

This way, we can support most of the existing fastify plugins, like the Passport one that @Gabrola mentioned.
All of the hooks' request object will also have the tsRestRoute like the express one to keep consistency.
Thoughts? @oliverbutler

Can the passport plug-in be used without this api? Just wondering as this feels just like we've got a api for running code, just before other code - which just sounds like an early return with extra steps no?
You guys are the expects here, more than happy to take your ways.
( @baryla worry about grand fathering in our api design, other than the Nest lib we need to give some love to our APIs)

The hook examples I gave would be direct mappings to what fastify supports. They have a bunch of request hooks and 2 of them are preValidation and onRequest so it wouldn't necessarily be an early return but actually what the user would provide here would be "piped" into the request instance.
What this would in theory look like, with the passport plugin;
import fastifyPassport from '@fastify/passport';

const app = fastify({ logger: false });
app.register(fastifyExpress);

server.register(fastifyPassport.initialize());

const router = s.router(contract, {
  getPosts: {
    hooks: {
      preValidation: [
        fastifyPassport.use(new SomeStrategy(...), { failureRedirect: '/login' }),
        // .. more pre validation hooks here
      ]
    },
    handler: async (tx) => {
      return {
        status: 200,
        body: []
      }
    }
  }
});

Does that make sense?

@baryla yup, that looks perfect! And I'd guess we'd have the a globalHooks option too for the entire router. Also this is probably going to bring the internal logic to be much more similar to the express implementation where we add the middleware directly with the route inside the fastify.route({ ...hooks, handler: ... }) just like in express app.get(path, ...handlers)
Also on a quick glance it doesn't seems like we can even what we want with middleware, the way it's like in the code currently app.use(...middleware).route(...) seems to indicate that we are adding the route's middleware to the entire app not just specific routes. But based on docs here, you need to use app.use(path, ...middleware) to add middleware to specific paths and it does not support routes with parameters or wildcards. So what we're offering with express wouldn't be possible unless we hack up the middleware implementation even further to support our use-case with route-specific middleware.

@oliverbutler @Gabrola what do you guys think about the changes from middlewares to hooks?

It would be awesome to see the option to attach middleware at different stages.
In fastify we have
Request/Reply Hooks
- onRequest
- preParsing
- preValidation
- preHandler
- preSerialization
- onError
- onSend
- onResponse
- onTimeout
- onRequestAbort
- Manage Errors from a hook
- Respond to a request from a hook
Application Hooks
- onReady
- onClose
- preClose
- onRoute
- onRegister

https://fastify.dev/docs/latest/Reference/Hooks/
Now I am trying to attach middleware onError but in docs, there is described only preHandler in the sentence

These run before any route-specific middleware.

https://ts-rest.com/docs/express/middleware

hi @baryla , do you mind fixing the merge conflict?

Gonna close for now, its been a hot while since this was active


Google Maps integration

Jiren,
Thank you so much for adding the Google Maps to the filtering. I LOVE the hover effect on the map--hadn't even thought of that. I almost have it where I need it, but am in a bit over my head to finish it up. Would you mind taking a look at my code? What's the best way to post? On github or for via email?
Thanks,
Catherine

You can send me code way you prefer.
------Original Message------
From: CatherineMortali
To: Jiren Patel
Subject: [filter.js] Google Maps integration (#3)
Sent: Jan 13, 2012 6:08 AM
Jiren,
Thank you so much for adding the Google Maps to the filtering. I LOVE the hover effect on the map--hadn't even thought of that. I almost have it where I need it, but am in a bit over my head to finish it up. Would you mind taking a look at my code? What's the best way to post? On github or for via email?
Thanks,
Catherine

Reply to this email directly or view it on GitHub:
https://github.com/jiren/filter.js/issues/3
Sent from BlackBerry® on Airtel

Jiren,
Thanks so much. I'm attaching the basic code for the Results page with filtering.
When I started working on the project last week, I couldn't get the demo filter.js code that's on gitHub to work using multiple categories. So, I looked at one of the GoodInKind pages that had multiple category filters and integrated that code. Now, I've got a filter.js code that's out of date, I'm afraid, and having troubles integrating it with the newest code that you've posted.
This is a nonprofit project for County-based Resources in NJ.
All of my filtering is working perfectly, maps are loading quick and accurately, but I'm not able to get to some of your "new features":

Filter a category, reflect changes on the map.
Hover over Resource Name to show marker on map.
Multiple Address mapping
Each resource has a primary address IF it is a physical location (some resources are events or links) but MAY have up to 2 additional addresses. I've included the primary lat/lng in the JSON file as well as a "locations" tag that shows all of them. Ideally, I'd like to check for those additional addresses and tie them to the markers using ID_Address_ID, so that they're unique for marker labeling and hover event.
List City Names
I looked on the dealacre site and saw how you listed the location name using the "getcityaddress" function. Looks straightforward to me, but I can't get it to work. I have some extra code in there from the GoodInKind site that may not be necessary that's causing the conflict?

I really appreciate your help and your time with this. I need to go live this weekend and would love to integrate these features in the launch if you have the time to look at it. I'd be happy to pay a consulting fee as well. This is SO useful, you have no idea!
Best regards,
Catherine

Catherine Mortali
WaterSpout Consulting
555 Cedar St
Walpole, MA 02081
(508) 446-1143
skype: catherine.mortali
On Jan 12, 2012, at 11:41 PM, Jiren Patel wrote:

You can send me code way you prefer.
------Original Message------
From: CatherineMortali
To: Jiren Patel
Subject: [filter.js] Google Maps integration (#3)
Sent: Jan 13, 2012 6:08 AM
Jiren,
Thank you so much for adding the Google Maps to the filtering. I LOVE the hover effect on the map--hadn't even thought of that. I almost have it where I need it, but am in a bit over my head to finish it up. Would you mind taking a look at my code? What's the best way to post? On github or for via email?
Thanks,
Catherine

Reply to this email directly or view it on GitHub:
https://github.com/jiren/filter.js/issues/3
Sent from BlackBerry on Airtel

Reply to this email directly or view it on GitHub:
https://github.com/jiren/filter.js/issues/3#issuecomment-3475450

Thank you. I just sent it via email.


Run several mesos-consul in parallel

Hi,
I couldn't find in the documentation whether it is ok to run several mesos-consul in parallel ? is there no risk of duplicated services/splitbrain or whatever ?
Thanks
Chris

You probably should only run one process, as it connects to the consul daemons on each system to register them.  There's really not a big risk of split brain in the process itself, it queries mesos and then registers the tasks on each consul node.

Yeah, but then this becomes a single point of failure :-/ Any way around that ?

We usually have run it via Marathon, so that it gets restarted if it dies. If you want stronger HA, maybe look at implementing a lock among multiple mesos-consul process via consul locks.

Well that is exactly what I do as well :-)
I was just worried of the bootstrapping problem in case of failure, since I generate mesos' whitelist from that.
Thanks !

We came to the same conclusion here.  Run one, and let Marathon deal with restarting it.  If it dies, you might wait a few seconds before a replacement starts up, which is okay because the poll cycle is a minute anyway.  We thought about multiple instances and decided against it.


No tweets collected past 10 days ago

Issue Template
Please use this template!
Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

[x] Python version is 3.6;
[x] Using the latest version of Twint;
[x] I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran

Please provide the exact command ran including the username/search/code so I may reproduce the issue.

c = twint.Config()
c.Until = '2018-10-19'
c.Search = "Trump"
# Run
twint.run.Search(c)

Description of Issue

Please use as much detail as possible.

Since this morning, Twint doesn't return any results past 10 days ago (Oct 19). The above command returns nothing. If I omit the c.Until line, it gets data until Oct 19 and then stops. Just last night I was getting tweets back to 2011. Upgraded to the latest ver this morning and got the same result.
Environment Details

Using Windows, Linux? What OS version? Running this in Anaconda? Jupyter Notebook? Terminal?

Python 3.6.7, IPython 2.4.1 and regular Python, Ubuntu Server 16.04.3 LTS

Update: The command works past 10 days from my Windows machine, but not Ubuntu. Still troubleshooting...

Can you share some more info on what happens when you try to run it on Ubuntu?

After the last command, it just returns me to a prompt. Output looks like this (using my Twitter username so it'll run faster):
In [22]: c = twint.Config()

In [23]: c.Until = '2018-10-19'

In [24]: c.Search = "@dfreelon"

In [25]: # Run

In [26]: twint.run.Search(c)

In [27]:

Works for me with no errors.
How did you install module?

I used pip3 install --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint
pip3 install twint installs a version with the #250 bug.

No problems with latest commit fe0fbad. Can you make sure your latest commit is fe0fbad?

Here's what I got:
freelon@ccrl:~/pytest/twint$ git log
commit fe0fbadc884150d9f65f885f2cf24ec6f3fd2486
Author: Francesco Poldi<EMAIL_ADDRESS>Date:   Mon Oct 29 18:56:10 2018 +0100

If it's just me, I'm happy to close this and work on it on my own, but it seems weird that the issue would just start all of a sudden... anyway, thanks for all your help and for the great program.

@dfreelon what makes things more strange is that in the past few hours, no changes were applied to the scraping mechanism
Also there are not any platform specific functions or settings, for us is difficult to replicate the issue and thus debug it

Yes I'm sure it must be my Ubuntu system--I just tried it on my Mac and
it works fine. Probably just something misconfigured. Thanks again for
all your help, much appreciated. Best, /DEEN
On 10/29/2018 6:49 PM, Francesco Poldi wrote:

@dfreelon https://github.com/dfreelon what makes things more strange
is that in the past few hours, no changes were applied to the scraping
mechanism
Also there are not any platform specific functions or settings, for us
is difficult to replicate the issue and thus debug it
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/twintproject/twint/issues/268#issuecomment-434109603,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGUFFwZP5nDPab_vBIUE4o0WdMgO-jm4ks5up4YAgaJpZM4X_Xw2.

--
Deen Freelon, Ph.D.
Associate Professor
School of Media and Journalism, UNC-Chapel Hill
http://dfreelon.org | @dfreelon https://twitter.com/dfreelon |
https://github.com/dfreelon


question

why i can't train a model that have open-voc ablility .
and RemoteCLIP model can't detect anything .
please help me

just use your code you give
python tools/train.py configs/visdrone_step1_base.py
python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model faster-rcnn
tools/test.py configs/visdrone_step1_base.py work_dirs/visdrone_step1_base/iter_80000.pth
And another question that  -----    what's the purpose of  the merge weight  and the self-train ?

just use your code you give
*python tools/train.py configs/visdrone_step1_base.py
*python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model faster-rcnn
*tools/test.py configs/visdrone_step1_base.py work_dirs/visdrone_step1_base/iter_80000.pth
And another question that  -----    what's the purpose of  the merge weight  and the self-train ?

and  the config file (12b-10k), the last line , i  change it  to {  load_from = 'checkpoints/RemoteCLIP-RN50.pt'   } .  is that  right? or maybe the  {new_model3.pth  }  that  i  trained

can you leave your email?

and  the test output is  the mAP just have 21%, it can't detect the unseen class .     i think there is something wrong

just use your code you give python tools/train.py configs/visdrone_step1_base.py
python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model faster-rcnn
tools/test.py configs/visdrone_step1_base.py work_dirs/visdrone_step1_base/iter_80000.pth
And another question that ----- what's the purpose of the merge weight and the self-train ?

the command for merge_weights.py should be: (#3 )
python tools/merge_weights.py --clip_path checkpoints/RemoteCLIP-RN50.pt --base_path work_dirs/visdrone_step1_base_openvoc/iter_80000.pth --save_path checkpoints/new_model3.pth --base_model soft-teacher
and the last line in your config file:
load_from = 'checkpoints/new_model3.pth'
then, self-training
python tools/train.py configs/visdrone_step2_castdet_12b_10k.py

My email<EMAIL_ADDRESS>

this output  is  by  the  step1 with openovc   and  got 80000.pth，
then merge-weight with  Remoteclip-rn50  to the  newmodel.pth
next self-training with step2 config file  and got 10000.pth
test  config  is  step2 config (12b-10k)
output is mAP 0.387
how to make 0.40 ?  in your paper  (how the Novel to be pre-known)?

ok，thanks，i got the mAP with 0.387

I also encounter zero-shot performance of 0 during testing, with an overall mAP of 0.24. How did you resolve this issue?

Train Step 1
During the merge, remove the state_dict and then merge
Provide the path to the merged model and train Step 2

this output is by the step1 with openovc and got 80000.pth， then merge-weight with Remoteclip-rn50 to the newmodel.pth next self-training with step2 config file and got 10000.pth test config is step2 config (12b-10k) output is mAP 0.387 how to make 0.40 ? in your paper (how the Novel to be pre-known)?

just use this  way to train and test


[OpenAI] Fixed ChatRequestUserMessage Deserialization Issue

One Class can only have one JsonCreator annotation. If provided more than one JsonCreator, jackson-databind will pick the later one by default but still caused confusion to jackson-databind.

API change check
API changes are not detected in this pull request.


Fix builtin default cost dependents on migration

Problem
Some of builtin programs are being migrated to sbpf, while the rest is being planned for too. Once a builtin program is migrated in a cluster, it's "default cost" defined in BUILTIN_INSTRUCTION_COSTS should be changed to 0.
Summary of Changes

add public function to fetch correct default cost for builtin program, based on migration feature status
add BuiltinCost struct to track migration feature ID
add developer notes to remind to add migration feature ID whenever it becomes available.

Fixes #

@tao-stones you updated benchmark code for the builtin costs.
Can you post the output of master vs this branch?
Does the result change much if you include the change from #3756

@tao-stones you updated benchmark code for the builtin costs.
Can you post the output of master vs this branch? Does the result change much if you include the change from #3756

Master - simple AhashMap.get, without checking feature
test bench_hash_find ... bench:      15,837.89 ns/iter (+/- 220.36)

Master w #3756
test bench_hash_find ... bench:      15,621.23 ns/iter (+/- 306.84)

 This branch w #3756 - Checks Option and is_active()
running 2 tests
test bench_hash_find_builtins_migrated     ... bench:      21,714.63 ns/iter (+/- 102.63)
test bench_hash_find_builtins_not_migrated ... bench:      21,373.65 ns/iter (+/- 81.10)

backports?


detoxURLBlacklistRegex in launchArgs not working (always nil)

[X] I have tested this issue on the latest Detox release and it still reproduces

Description
No matter what value I set detoxURLBlacklistRegex to in launchArgs when launching the app, it is always nil when picked up in the native code.
If I replace [options arrayForKey:@"detoxURLBlacklistRegex"];with<EMAIL_ADDRESS>my tests run as expected. I've tried setting the values to all of the following:

The example: '\\("http://<IP_ADDRESS>:19001/onchange","https://e.crashlytics.com/spi/v2/events"\\)'
'\\(".*"\\)'
'\\(.*\\)'
['.*']

Reproduction

Update detox/ios/Detox/DetoxManager.m to print out "blacklistRegex"
Run detox tests using detoxURLBlacklistRegex in launchApp (as mentioned in the docs)

Expected behavior
The value should be set to what is passed in detoxURLBlacklistRegex
Environment (please complete the following information):

Detox: 15.1.2
React Native: 0.61.5
Node: 12.14.1
Device: iPhone 8 (tried on multiple)
Xcode: 11.3.1
iOS: 13.3
macOS: 10.14.5

Logs
If you are experiencing a timeout in your test

[X] I have followed the instructions under Lower-level Idling Resources Debug and am providing the log below:

Omitting the log for brevity. It shows a lot of App State -> busy Waiting for network requests to finish: ... logs while launching

Any news on this?

@LeoNatan

This likely broke after the rewrite of arguments. It means this feature is broken.

Is there a specific version of Detox we can use in the meantime until this gets sorted out?

I'm sorry, I'm not sure which version introduced this issue. It was a while back. Contributions are welcome.

@michaelgmcd when you escape those " characters,
what is the output you get back from your NSProcessInfo.processInfo.arguments sent through websockets?
is there an easy way to set the logging through websockets up?

Hey @LeoNatan @renanmav @Durisvk
if using detox on v14.0.0
and using:
await device.setURLBlacklist(['.*googleapis.*']);
works for me

@jaburx I'm missing in the docs, when is the right time to call await device.setURLBlacklist(['.*googleapis.*']);
is it before detox.init? after detox.init?
the problem is I'm using detox for Expo ejected app and it gets stuck on init and on setURLBlacklist as well

Sending the param as launch argument and using setURLBlacklist are two different things. The bug is in the former, while the latter passes the data over the web socket.

Its after init

Hey Leo, we know that, but u guys were asking for a way to run it.

But the suggestion doesn't work here, because if an app makes requests on launch, Detox will be stuck waiting for them, unable to reach idle state, so it can process the setURLBlacklist() call.

Hello, I dug into this issue and couldn't find where it was altered. I played with NSUserDefaults and launch args in XCode, and in fact there's no problem with detox.
You can use this function to generate the correct value of detoxURLBlacklistRegex from a list of domains:
const getDetoxURLBlacklistRegexFromDomains = domains =>
  '(' + domains.map(domain => `\\".*${domain}.*\\"`).join(',') + ')';

This generates the following CLI arg, with your example @michaelgmcd :
 -detoxURLBlacklistRegex "(\".*codepush.azurewebsites.*\",\".*clients3.google.*\",\".*googleapis.*\",\".*app-measurement.*\",\".*crashlytics.*\")"
Happy to do a PR to add a more friendly API if need be.

@Minishlink, it works! Weird that I tried a very similar function that didn't. I think a friendlier API would make sense. Maybe just accept an array of url regex strings?

Is there any solution that doesn’t involve diving into the workings of Detox, and just using the
await detox.init(config, { launchApp: false, detoxURLBlacklistRegex: ""});
If there is (having kept a close eye on this bug) I would be very grateful to know what regex I would need to blacklist pub-nub domains. I.e. ps1.pndsn.com, ps2.pndsn.com, ps3.pndsn.com, ps22.pndsn.com etc etc
I know it would seem straightforward, but nothing seems to work.
Thanks

@mckenzit Have you seen my comment?
You would then have to do :
const detoxURLBlacklistRegex = getDetoxURLBlacklistRegexFromDomains(['pndsn.com']);
await detox.init(config, { launchApp: false, detoxURLBlacklistRegex });

What's the status here, then? Should we pin or close?


Avoid identifiers reserved by Ruby

"prefix rb shows that it belongs to ruby".
[RHG]

rb_ is a prefix to prevent exported symbols in libruby from
conflicting with symbols in other libraries, in particular,
dynamically loaded Ruby extension libraries.
So the prefix is reserved by Ruby by convention.
For gRPC, simply "grpc_" would be a better prefix.
References

[RHG]: Minero Aoki. "The Ruby Hacking Guide" (2002), chapter 4

@yugui: Thanks for this! Could you make the prefix  grpc_rb_ instead of grpc_ ?
This is

to minimize the risk of clashes with names in the C core
make it easier to identify ruby extension symbols when debugging

@tbetbetbe
Sure.  I'll replace the prefixes as you said.
Also may I ask a question? I wonder if we need to export some symbols, e.g. grpc_rb_cEvent.
These variables need to be exported only if:

you need to refer the variable in another compilation unit in your extension library; or
you want other extension libraries to directly access to your implementation details for some reasons, e.g. performance.

I guess this is not the case for these symbols in this extension.  If so, you can keep them in static storage and I'm happy to write another pull request.
What do you think?

Replaced the prefix with "grpc_rb_". Please take another look.

Thanks, LGTM

I guess this is not the case for these symbols in this extension. If so, you can keep them in static storage and I'm happy to write another pull request.
What do you think?

This would be a great improvement - thanks for offering, I will look forward to the next pull request.

Thank you for your pull request. Before we can look at your contribution, we need to ensure all contributors are covered by a Contributor License Agreement.
After the following items are addressed, please respond with a new comment here, and the automated system will re-verify.

User @yugui isn't covered by a CLA. They will need to complete the form at https://identity.linuxfoundation.org/projects/cncf

Regards,
CLA GitHub bot


Releases of fluxd look like failures because they kill fluxd

In Slack: fluxy/fluxd: Regrade due to "Release latest images to fluxy/fluxd": failed: unexpected EOF

Fixed by #307.


fix fix parsing fails when json key string is empty.

Proposed changes
This is a very subtle bug that can only be reproduced in a non-AVX2 environment.
// USE_AVX2 = false
mysql [(none)]>mysql [(none)]>select cast('{"":1, " ":"v1"}' as json);
+----------------------------------+
| cast('{"":1, " ":"v1"}' as JSON) |
+----------------------------------+
| NULL                             |
+----------------------------------+

//USE_AVX2 = true
mysql [(none)]>select cast('{"":1, " ":"v1"}' as json);
+----------------------------------+
| cast('{"":1, " ":"v1"}' as JSON) |
+----------------------------------+
| {"":1," ":"v1"}                  |
+----------------------------------+

run buildall

run buildall


Chroma subsampling

Thanks for your demo examples, especially the one exposing all the options, very helpful!!!
https://3bl3gamer.github.io/wasm-mozjpeg/examples/settings/
Do you have any insights regarding the chroma subsampling parameters?
https://kraken.io/docs/chroma-subsampling
Playing around with things it seems like...
h=1, v=1 => 4:4:4 (no chroma subsampling)
Also, I note that squoosh.app doesn't expose h and v parameters for mozjpeg, just a single chroma subsampling parameter, as well as an "auto" option which is a mystery as to what they are using.
Edit:  OK, I think h and v parameters correspond to standard n x n terminology, so...
h=1, v=1 => 1x1 => 4:4:4
h=2, v=1 => 2x1 => 4:2:2
h=2, v=2 => 2x2 => 4:2:0
https://www.impulseadventure.com/photo/chroma-subsampling.html
It is still a mystery to me what squoosh is doing with "auto", perhaps they use 1x1 or 2x1 for higher jpeg quality, and 2x2 for lower jpeg quality.  There seems to be a discontinuous jump in squoosh file size at quality 90 so maybe that's what's going on.

h and v parameters correspond to standard n x n terminology,

Yes, it should. And you can even increase h and v up to 4.

discontinuous jump in squoosh file size at quality 90

MozJpeg by default sets sub-sampling to 2x1 for quality >= 80 and 1x1 for quality >= 90:
https://github.com/mozilla/mozjpeg/blob/c78b82bbb12ed6d41e2f896631aa3bab7a7f8c92/rdswitch.c#L557-L565
Maybe squoosh.app do the same.


I tried to build a project, but cgo code kept failing
code demo:
`package main
import(
"fmt"
"gopkg.in/olebedev/go-duktape.v3"
)
func main()  {
ctx := duktape.New()
ctx.PushGlobalGoFunction("log", func(c *duktape.Context) int {
  fmt.Println(c.SafeToString(-1))
  return 0
})
ctx.PevalString(`log('Go lang Go!')`)

}`
What did you see happen?
In command line:
go build -o main2 main.go
runtime/cgo
In file included from _cgo_export.c:4:
cgo-gcc-export-header-prolog:31:14: error: size of array ‘_check_for_64_bit_pointer_matching_GoInt’ is negative
What did you expect to see?
compile fine when GOARCH is set to host arch, but fail when cross compile to arm64
what can I do to fix this?

Well, the markdown sure messed up,  hope it doesn't affect you understanding what my problem is XD

Similar to this issue, but I believe I set the cross compiler correctly
https://github.com/golang/go/issues/38693

that's an arm32 toolchain.
Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.
For questions please refer to https://github.com/golang/go/wiki/Questions


The problem of two gpus training

Hello, I want to train the model with two gpus, I changed os.environ['CUDA_VISIBLE_DEVICES']='0,1' and gpus=2, there is a RuntimeError:Distributed package doesn't have NCCL built in problem, please have any suggestions?

Hi @cici-999, how about trying CUDA_VISIBLE_DEVICES=0,1 python tools/train_net.py ...

Hi @cici-999, how about trying CUDA_VISIBLE_DEVICES=0,1 python tools/train_net.py ...

I tried the method you mentioned and I don't get RuntimeError, but it still shows 1 gpu for training.


Compiler generates broken code by declaring variables out of order.

I have native modules in my app.
When I compile, the output contains the line
Elm.Native.StartSeed = {};
ahead of
var Elm = {};
which inevitably leads to

Uncaught TypeError: Cannot read property 'Native' of undefined

Using Elm 0.17.1, any browser.

Update: although there were no problems preventing compilation, I decided to use --warn and clean up any outstanding issues. They consisted of some unused imports as well as some values that had no type annotations.
After I corrected these warnings, the compiled output is fine!

Huh. Would it be possible to get this down to a http://sscce.org/? At this point all we have is,
"if something is weird with unused imports, compilation breaks".

I'll try to create an sscce when I have time, probably later this week. The mix was: native modules, unused imports, missing type annotations.

Open an new issue if you make an SSCCE. No 0.17 compiler should be generating code like you shared, so I suspect the issue is that your PATH had multiple compilers and the wrong one was being invoked.


Ref: remove await and unused promise for tracking function

Also improve error message.

Closed in favor of https://github.com/bitpay/bitpay-app/pull/256


Tesseract Error

input:
os.environ['TESSDATA_PREFIX'] = "C:\Program Files\Tesseract-OCR\tessdata"
custom_config = r'--tessdata-dir "/content/kan.traineddata" -l kan'
custom_config = r'-l kan --psm 6'
for img in img_list:
im = cv2.imread(f'/content/sample_data/sample_images/{img}')
img_rgb = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)
output = pytesseract.image_to_string(img_rgb, config=custom_config)
print(output)
dictionary = {'img_filename':img, 'ocr_output':str(output)}
df_output = df_output.append(dictionary, ignore_index=True)
output:
TesseractError: (1, 'Error opening data file C:\Program Files\Tesseract-OCR\tessdata/kan.traineddata Please make sure the TESSDATA_PREFIX environment variable is set to your "tessdata" directory. Failed loading language 'kan' Tesseract couldn't load any languages! Could not initialize tesseract.')

Please respect the guidance for posting issue:

We do not support 3rd party projects (pytesseract). Replicate the problem with tesseract executable
provide all required information

Your code have mistakes.
os.environ['TESSDATA_PREFIX'] = "C:\Program Files\Tesseract-OCR\tessdata"
custom_config = r'--tessdata-dir "/content/kan.traineddata" -l kan'
custom_config = r'-l kan --psm 6'

instead of:
custom_config = r'--tessdata-dir "full/path/to/tessdatadir"'
custom_config += r'-l kan --psm 6'

In the path to Tesseract's data dir, don't add the file name (kan.traineddata).
If you still have issues, read what @zdenop wrote.


Microsoft Defender for Cloud enablement rules - #1632

PR Summary
Check each Microsoft Defender for Cloud Pricing plan is Standard
PR Checklist

[ ] PR has a meaningful title
[ ] Summarized changes
[ ] Change is not breaking
[ ] This PR is ready to merge and is not Work in Progress
Rule changes

[ ] Unit tests created/ updated
[ ] Rule documentation created/ updated
[ ] Link to a filed issue
[ ] Change log has been updated with change under unreleased section

Other code changes

[ ] Unit tests created/ updated
[ ] Link to a filed issue
[ ] Change log has been updated with change under unreleased section

@jdewisscher Also don't forget to add an entry in the change logs for all the great work. https://github.com/Azure/PSRule.Rules.Azure/blob/main/docs/CHANGELOG-v1.md


Training embeddings on SD2.1 error

When i try to preprocess images for embedding it returns me this error on SD2.1-768
ValueError: The following model_kwargs are not used by the model: ['encoder_hidden_states', 'encoder_attention_mask'] (note: typos in the generate arguments will also show up in this list)

Could be the solution but not a good one - https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1458

try !pip install transformers==4.19.2

!pip install transformers==4.19.2

New to colab, where do I insert this command?

!pip install transformers==4.19.2

New to colab, where do I insert this command?

You can add a new code cell, at the command and then run the cell.


No module named "CloudFlare"

Traceback (most recent call last):
  File "/home/dietpi/cfspflat/app.py", line 3, in <module>
    from cfspflat.cli import main
  File "/home/dietpi/cfspflat/cfspflat/__init__.py", line 7, in <module>
    from .cf_dns import TXTrec
  File "/home/dietpi/cfspflat/cfspflat/cf_dns.py", line 2, in <module>
    import CloudFlare
ModuleNotFoundError: No module named 'CloudFlare'

Module "cloudflare" is installed and working.

This is due to an API change of the Cloudflare Python API. See #3 for a fix.

updates added


feat: add customizable generic oidc login button

What type of PR is this?
feature
What does this PR do / Why do we need it:
This PR allow usage of generic OIDC with customizable name (linked to zot https://github.com/project-zot/zot/pull/1691)
Will this break upgrades or downgrades. Has updating a running cluster been tested?:
Won't break on upgrade / downgrades
Does this change require updates to the CNI daemonset config files to work?:
no
Does this PR introduce any user-facing change?:
If enabled on backend, it allows the display of a customizable login button
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Integration test patch is merged, you can rebase the branch

@raulkele done :)

There is still a debate on zot about keeping dex keyword.
I'll keep the conversation unresolved until decision is made and act accordingly.

@babs, @raulkele, I have merged https://github.com/project-zot/zot/pull/1691 on backend side.

I'll adapt the PR accordingly, rebase and squash

After rebase, from my point of view, we can merge this PR

Rebase done with removal of DEX reference as discussed


Catch reallocarray's overflow failure and warn

       However,  unlike  that  realloc()  call, reallocarray() fails safely in the case where the multiplication
       would overflow.  If such an overflow occurs, reallocarray() returns  NULL,  sets  errno  to  ENOMEM,  and
       leaves the original block of memory unchanged.

ok turns out it's kinda hard to access errno since gcc does weird things with errno:
   0x7ffff7dddde0 <__errno_location+0> endbr64
   0x7ffff7dddde4 <__errno_location+4> mov    rax, QWORD PTR [rip+0x198065]        # 0x7ffff7f75e50
   0x7ffff7ddddeb <__errno_location+11> add    rax, QWORD PTR fs:0x0
 → 0x7ffff7ddddf4 <__errno_location+20> ret

it calls this func to fetch the __errno_location. Last thing we need is yet another symbol to resolve tho...

consider tracking the brk syscall inside the allocator func to see where errno comes from? https://stackoverflow.com/a/61601162


Matrix multiplication in cpp

Is your feature request related to a problem? Please describe.
It is problem of backtracking.
Describe the solution you'd like
In this problem, instead of focusing on how to multiply two matrix. We find that how many steps will be required to calculate multiplication of the matrices.
Do you want to work on it
Yes, i want to work on this problem in c language.

/assign

Please assign this to me in Python

Please assign this to me in C++.I am a GSSOC'21 participant.

Please do assign this to me.


Fix/badges

Summary by CodeRabbit

Chores

Integrated CodeClimate coverage reporting in the CI/CD pipeline to enhance code quality tracking.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1a64297) 100.00% compared to head (0b8b5a1) 100.00%.
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main        #5   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            1         1
  Lines           20        20
  Branches         4         4
=========================================
  Hits            20        20

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


I don't understand HideStrings setting

Hi
Can someone advice me how to hide strings ?
private const string MyString = "Hallo World"; will only translated to
private const string A = "Hallo World";
I guess I understood something wrong. Anyhow , is it possible to obfuscate strings  ?
Wolfgang

You are not doing anything wrong. Strings are obfuscated at the call site by replacing the ldstr instruction with some code that returns the real string. Class-level constants, however, can't be obfuscated, since they are basically a blob of data, and not code.
However, I think it would make sense to remove class-level private constants from the assembly entirely...

Moves strings from consts to the method. And this is whole protection? What is a user's profit?

@geroz9 The profit is that strings are not readable as plain text anymore.

IlSpy (2.3.1) displays these strings in method's body and it is still readable.

Can you please post a screenshot? If that's the case, that might be a bug.
On Sun, Jun 3, 2018, 19:33 geroz9<EMAIL_ADDRESS>wrote:

IlSpy (2.3.1) displays these strings in method's body and it is still
readable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lextm/obfuscar/issues/122#issuecomment-394177944, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA4rJnjhH1IUjZcyAQwITsPuD6Mh47MGks5t5B3ZgaJpZM4P5DSp
.

public class AClass {
public string aString;
public void AMethod() {
this.aString = "This is the test string";
Console.WriteLine(this.aString);
}
}
Obfuscar output
Note that Rollbar API is enabled by default to collect crashes. If you want to opt out, please run with -s switch
Loading project V:[Temp3]\ConsoleApp3\ConsoleApp3\bin\Debug\netcoreapp2.0\out\ocar.xml...Processing assembl
: ConsoleApp3, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Loading assemblies...Extra framework folders: Done.
hiding strings...
Renaming:  fields...parameters...properties...events...methods...types...Done.
Saving assemblies...Done.
Writing log file...Done.
Completed, 0,59 secs.
IlSpy
using System;
namespace A
{
public class a
{
public string A;
public void A()
{
this.A = "This is the test string";
Console.WriteLine(this.A);
}
}
}
Sorry for formatting but editor seems to be uncontrollable for me now.

I'll check. But... Are Issues gone from this repo? I can't seem to open
this issue in the browser, are you seeing the same thing?
03.06.2018 20:12, geroz9 пишет:

public class AClass {
public string aString;
public void AMethod() {
this.aString = "This is the test string";
Console.WriteLine(this.aString);
}
}
Obfuscar output
Note that Rollbar API is enabled by default to collect crashes. If you
want to opt out, please run with -s switch
Loading project
V:[Temp3]\ConsoleApp3\ConsoleApp3\bin\Debug\netcoreapp2.0\out\ocar.xml...Processing
assembl
: ConsoleApp3, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null
Loading assemblies...Extra framework folders: Done.
hiding strings...
Renaming:
fields...parameters...properties...events...methods...types...Done.
Saving assemblies...Done.
Writing log file...Done.
Completed, 0,59 secs.
IlSpy
using System;
namespace A
{
public class a
{
public string A;
public void A()
{
this.A = "This is the test string";
Console.WriteLine(this.A);
}
}
}
Sorry for formatting but editor seems to be uncontrollable for me now.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lextm/obfuscar/issues/122#issuecomment-394180473,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA4rJrFwsfWmPZcoXFcDPSK2nkT1EKVoks5t5CcYgaJpZM4P5DSp.

@geroz9
Can't reproduce. This is what I get:
	public class AClass
	{
		public string aString;

		public void AMethod()
		{
			this.aString = D3D7049F-97A1-4235-9553-B26FF04F91C5.A();
			Console.WriteLine(this.aString);
		}
	}

Can you show the XML configuration file? What Obfuscar version do you use?


Cannot Install in Ubuntu 20.04 (Focal)

Error
Cannot install 'libavcodec-extra'
Steps to replicate:

Install Gdebi Package installer in Ubuntu 20.04.3 LTS
Use the above to install the following file :https://github.com/clangen/musikcube/releases/download/0.96.10/musikcube_0.96.10_ubuntu_focal_amd64.deb

Dang, apologies for the issues here. I'll see about getting some updated packages built in the next day or two.

Actually, according to Ubuntu's website, libcodec-extra does exist: https://packages.ubuntu.com/focal/libavcodec-extra
I guess maybe you have to enable the universe deb repository in your sources.list?

This should also be solved by using a new "standalone" build introduced in the 0.96.11 release here: https://github.com/clangen/musikcube/releases/tag/0.96.11.


MOBILE-242 | Password Reset is failing from mobile

Out of curiosity, where is the bug in the pre-existed code?

Out of curiosity, where is the bug in the pre-existed code?

The original bug was that we're calling getConfigString("passwordChanged.subject") and it was crashing because that path didn't exist. Instead, it should've been getConfigString("mintingNotifications.passwordChanged.subject") .


Parsing and tokenizing email addresses in project members form

After some experimentation I found that project member emails must be formatted as
"email1, email2"
with a comma-space between each address.
I initially tried to input emails to the form using a space-separated list, which it accepted without error, but it did not share the project to other people. I realize now that it treated that space-separated list as one very long email address.
Can we please add functionality to parse the email address list by

spaces
comma
tab
semicolon

We view this as a high priority improvement.
As a secondary goal, if we could tokenize the email addresses after adding each that would be a very nice UI improvement, e.g. how gmail tokenizes email addresses -

fixed with checkin to project_view_plotly branch.  Not yet deployed to prod


Add files for release, fix tox - 111

[Auto generated]
Number: [#111]

qe-metrics(ZnPQe)[PR 111]: Closing issue for PR: Add files for release, fix tox.
PR was closed.


GPS infos not found

Hi @TechnikTobi, trying to extract GPS infos from a picture I know it has GPS infos, I don't get any. Code example:
use std::path::Path;

use little_exif::exif_tag::ExifTag;
use little_exif::metadata::Metadata;
use little_exif::u8conversion::U8conversion;

fn main() {
    let metadata = match Metadata::new_from_path(Path::new("resources/img/rose.jpg")) {
        Ok(m) => Some(m),
        Err(_) => None,
    };

    match metadata {
        Some(m) => {
            let tag = m.get_tag(&ExifTag::CreateDate(String::new())).unwrap();
            if let Some(_gps_tag) = m.get_tag(&ExifTag::GPSInfo(vec![1])) {
                println!("GPS tag found");
            } else {
                println!("No GPS tag found")
            }
            let endian = m.get_endian();
            let extracted = String::from_u8_vec(&tag.value_as_u8_vec(endian), endian);
            println!("{}", extracted);
        }
        None => (),
    }
}

Any idea, am I doing something wrongly or is it a bug?
Thanks a lot in advance.

Hi @sgubian, can you provide an example image so that I can reproduce this locally?

Hi @TechnikTobi, thanks for your message.
I have put the attachment here. Hope it keeps the EXIF data when uploading as an asset on Github.

Thanks a lot for you help.

The EXIF data is kept and I am able to read it, however there is no GPS data included. I checked this both using the macOS Preview app (which, to my mind, should be able to decode the GPS data from an iPhone 12 Pro image), as well as by looking through the image data using a hex editor.
How do you know that this image file has GPS data stored?

Sorry, this one was without 🙄. This one has:

I used this site to check:
https://exifinfo.org/
And if gives me this:

Alright, now I see it as well. I guess you are using the release version of little_exif, version 0.5.1? In that case, yes, it is a known problem that sometimes GPS tags like yours can't be read. The good news is that there is a new decoder in development for the release of version 0.6.0 (which is expected to be published some time by the end of this week). I tested the current beta version with your image file and all of the GPS tags can be read successfully.
So you can either switch already to the current beta version or wait a few days until the full version is released.

Many thanks @TechnikTobi ! I will for now use 0.6.0 from the repos in the meantime it gets to crates.io. Cheers and thanks for this nice contrib to Rust.

Version 0.6.0 has been released!


some problems about "uvicorn"

Every time I run "uvicorn --app-dir=deepdenoiser app:app --reload --port 8003", I got the error messages below
INFO:     Will watch for changes in these directories: ['/home/as/Work/tmp/deepdenoise/DeepDenoiser']
INFO:     Uvicorn running on http://<IP_ADDRESS>:8003 (Press CTRL+C to quit)
INFO:     Started reloader process [23817] using statreload
/home/as/anaconda3/envs/deepdenoiser/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/convolutional.py:414: UserWarning: `tf.layers.conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.Conv2D` instead.
  warnings.warn('`tf.layers.conv2d` is deprecated and '
/home/as/anaconda3/envs/deepdenoiser/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1719: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
/home/as/anaconda3/envs/deepdenoiser/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/normalization.py:308: UserWarning: `tf.layers.batch_normalization` is deprecated and will be removed in a future version. Please use `tf.keras.layers.BatchNormalization` instead. In particular, `tf.control_dependencies(tf.GraphKeys.UPDATE_OPS)` should not be used (consult the `tf.keras.layers.BatchNormalization` documentation).
  '`tf.layers.batch_normalization` is deprecated and '
/home/as/anaconda3/envs/deepdenoiser/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/core.py:268: UserWarning: `tf.layers.dropout` is deprecated and will be removed in a future version. Please use `tf.keras.layers.Dropout` instead.
  warnings.warn('`tf.layers.dropout` is deprecated and '
/home/as/anaconda3/envs/deepdenoiser/lib/python3.7/site-packages/tensorflow/python/keras/legacy_tf_layers/convolutional.py:1294: UserWarning: `tf.layers.conv2d_transpose` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.Conv2DTranspose` instead.
  warnings.warn('`tf.layers.conv2d_transpose` is deprecated and '
2021-09-19 18:12:01.694317: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-09-19 18:12:01.717938: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency:<PHONE_NUMBER> Hz
2021-09-19 18:12:01.718885: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55f47dc085a0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-09-19 18:12:01.718923: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-09-19 18:12:01.719037: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
2021-09-19 18:12:01.791547: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
restoring model /home/as/Work/tmp/deepdenoise/DeepDenoiser/model/190614-104802/model_95.ckpt
Connecting to k8s kafka
WARNING:kafka.conn:DNS lookup failed for quakeflow-kafka-headless:9092, exception was [Errno -3] Temporary failure in name resolution. Is your advertised.listeners (called advertised.host.name before Kafka 9) correct and resolvable?
ERROR:kafka.conn:DNS lookup failed for quakeflow-kafka-headless:9092 (AddressFamily.AF_UNSPEC)
WARNING:kafka.conn:DNS lookup failed for quakeflow-kafka-headless:9092, exception was [Errno -3] Temporary failure in name resolution. Is your advertised.listeners (called advertised.host.name before Kafka 9) correct and resolvable?
ERROR:kafka.conn:DNS lookup failed for quakeflow-kafka-headless:9092 (AddressFamily.AF_UNSPEC)
k8s Kafka connection error
Connecting to local kafka
ERROR:kafka.conn:Connect attempt to <BrokerConnection node_id=bootstrap-0 host=localhost:9092 <connecting> [IPv4 ('<IP_ADDRESS>', 9092)]> returned error 111. Disconnecting.
ERROR:kafka.conn:Connect attempt to <BrokerConnection node_id=bootstrap-0 host=localhost:9092 <connecting> [IPv4 ('<IP_ADDRESS>', 9092)]> returned error 111. Disconnecting.
local Kafka connection error
INFO:     Started server process [23819]
INFO:uvicorn.error:Started server process [23819]
INFO:     Waiting for application startup.
INFO:uvicorn.error:Waiting for application startup.
INFO:     Application startup complete.
INFO:uvicorn.error:Application startup complete.

Any reply and comment is appreciated.

Hi, that is normal. The errors are from kafka. This Interactive example is mainly for the QuakeFlow project.


Add native maxout op

Maxout op: opNum() isn't implemented, rendering it unusable currently.
Presumably we need a native implementation for this too.
https://github.com/deeplearning4j/nd4j/blob/e59d5f58942e283f4dc62169e193e22aefd1aa5a/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/transforms/MaxOut.java#L60

We could have a maxout vertex that does this once. Technically this doesn't have to be an activation function though it is as much.
https://github.com/deeplearning4j/libnd4j/issues/444


Twitter API: Map the raw data to internal entity types

The Twitter API responses are complex JSON objects, so we need to split that data into smaller objects.
Right now we have:
export type Tweet = {     id: string,     author_id: string,     text: string,     created_at: string,     attachments?: {         media_keys: string[]     },     public_metrics: TweetPublicMetrics;     user?: User, }
export type User = {     id: string,     name: string,     username: string,     url: string,     profile_image_url: string,     description: string,     created_at: string,     verified: boolean,     public_metrics: UserPublicMetrics,     entities: {         [key: string]: Record<string, Entity[]>,     }, }
We need a function that accepts the raw JSON response and maps its data to arrays of these types.
It returns a map in the form of:
{  tweets: Tweet[],  users: User[] }
We should export the function from './repos/twitter.ts'

I think we should rid of the nested objects in our types because the Redis JSON module doesn't support them...
We will level up the public metrics and the media keys, and we will JSON.stringify the entities fields because they are really complex object we cannot flatten.

Are you saying we should remove the public metrics and media key from our tweet data?

No, no -  not removing them. Just flatten the structure by getting the props of the nested objects and making them props of the top-level object, because Redis Json Module doesn't support nested objects


Add support for map properties

Context
Many properties of tasks or extension objects are best modeled using a Map. We already have ListProperty and SetProperty for collection-valued properties; this PR adds a MapProperty as well.
Contributor Checklist

[x] Review Contribution Guidelines
[x] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[x] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[x] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[x] Provide unit tests (under <subproject>/src/test) to verify logic
[x] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[x] Ensure that tests pass locally: ./gradlew <changed-subproject>:check

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

Thanks for the PR. The changes looks good and we'll merge them soon.


fix: repair wrong types passed to DataTag

This PR fixes the types that DataTag uses to define the type of data in the cache. It was initially using TData, but if select was used in queryOptions, this type would change to the transformed selected data, which is wrong.
The changes in this PR consist of using TQueryFnData instead, and add select to the type tests to ensure the correct value is being used, since the lack of this prop led to false test positives

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (3a127bf) 41.71% compared to head (d0e1299) 77.26%.
Report is 2 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #6699       +/-   ##
===========================================
+ Coverage   41.71%   77.26%   +35.55%
===========================================
  Files         179       54      -125
  Lines        7007      893     -6114
  Branches     1416      213     -1203
===========================================
- Hits         2923      690     -2233
+ Misses       3713      164     -3549
+ Partials      371       39      -332

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@TkDodo done


Better error handling for K8S multi-pod submissions

Referencing PR https://github.com/twitter/heron/pull/1970, we need to be able to better handle problems that may arise during a multi-pod submission. Since Kubernetes does not currently support submitting multiple pods for deployment in a single API call, a problem could arise where one of the containers fails to be added or destroyed and leave the topology in a bad state.
Possible solutions:

Figure out a way to rollback existing submissions
Use a different kind of deployment for K8S (possibly a Deployment object?). Would require changes where every container would have a common the same startup command -- which is impossible now since the container index is needed (which is different for every container)

This issue is 3 years ago, but i believe heron has more to do with this. To clarify one thing that, k8s supports multiple container within 1 pod, which means we can group up some into one to have cleaner setup.

Heron Kubernetes scheduler also now supports launching topologies using Kubernetes StatefulSets. I believe this issue has been satisfied with that functionality. There is still legacy code that launches the individual pods, but that most likely should be deprecated.
I'm closing the issue, but feel free to submit any new issues for features that are still lacking.


Bug in TextUtils.parseTimeStamp

From the looks of it parseTimeStamp doesn't seem to work.
https://github.com/UBOdin/mimir/blob/master/src/main/scala/mimir/util/TextUtils.scala#L37
It's replicating field 1 for nearly all entries.
Working on a test case and fix.

Fixed in 4e3633429302106aa6baf5ce9cb93f634ae2693f


feat: update in logging to console and Unify case usage, seperators and punctuation for logging

Signed-off-by: Mihir Mittal<EMAIL_ADDRESS>

This PR
Related Issues

[FEATURE] Pretty-print logging #237
[REFACTOR] Unify case usage, seperators and punctuation for logging #298

Please check this commit.

It looks like the PR is failing on linting and tests, please can you resolve these issues.
It also might be more appropriate to move the changes from commit 5081d69ca6f404d3c0a06ebacc006f63c16ef057 into a separate PR as these changes resolve a separate issue.

plz check now. I have solved two issues in this one #237 and #298.

Also, please run make generate-docs to update the documentation.

plz review it

@james-milligan plz see now

The OFO should be updated once this feature is available.
https://github.com/open-feature/open-feature-operator/issues/334

Thank you for your contribution @mihirm21! This feature will be available in the next release.


Why should we use CPU?

And if sb can transfer it into a GPU version, how?
If anyone knows, pls give a commit. Thx!

what do you mean by sb?
We mostly train on CPU because our networks are fairly small, and most of the time is still taken up by the simulation.

Oh, I understand. Thx for explaining very much!


Improve device usage within Dump loader command

There is currently no validation, if given slug is something like "Plip plop's plap", there will be an exception from Django, which is not very nice.
The command should have an option to fill the title when creating a new device.

Done in 0.6.2-pre.3


Error: Cannot open output data buffer

Thanks for your effort to this excellent repo.
I'm using the grobid service to build a website.
The server configuration:

CPU: Intel(R) Xeon(R) CPU E5-2620
Memory: 16.0 GB
OS: Windows Server 2008 R2 Enterprise x64

I meet this error. Is that due to not enough memory?
<IP_ADDRESS> - - [17/五月/2020:04:47:17 +0000] "POST /api/processFulltextDocument HTTP/1.1" 200 14333 "-" "Java/1.8.0_251" 2624
INFO  [2020-05-17 04:47:19,360] org.grobid.core.factory.GrobidPoolingFactory: Number of Engines in pool active/max: 1/10
WARN  [2020-05-17 04:47:24,302] org.grobid.core.lang.impl.CybozuLanguageDetector: Cannot detect language because of: com.cybozu.labs.langdetect.LangDetectExcept
ion: no features in text
INFO  [2020-05-17 04:47:27,293] org.grobid.core.factory.GrobidPoolingFactory: Number of Engines in pool active/max: 2/10
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\affiliation-address\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\affiliation-address\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\name\header\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\name\header\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\name\citation\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\name\citation\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\header\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\header\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\date\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\date\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\citation\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\citation\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\fulltext\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\fulltext\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\segmentation\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\segmentation\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\reference-segmenter\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\reference-segmenter\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\figure\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\figure\model.wapiti
[Wapiti] Loading model: "E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\table\model.wapiti"
Model path: E:\PDFAnalysisPlatform\grobid-0.5.6\grobid-home\models\table\model.wapiti
error: Cannot open output data buffer
        <No error>

> Task :grobid-service:run FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':grobid-service:run'.
> Process 'command 'C:\Program Files\Java\jdk1.8.0_251\bin\java.exe'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/5.4.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 62h 29m 16s
6 actionable tasks: 1 executed, 5 up-to-date

Hello @Licheng-Xu !
It's not a problem of memory (16GB is a lot for GROBID. The online demo of Grobid running on a 8GB, it is continuously overloaded but almost never goes OOM).
Are the tests working?

.gradlew test

Possibly the same issue as #436 and #462 and it's hard to test for me without a Windows machine :/
Maybe you could use the Docker container for ensuring something reliable on Windows?

I switched to the Docker container and it works well. Thanks a lot.


Autocomplete fails if next line has a comment

What version of Go are you using (go version)?
1.11
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home//.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home//go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home//src/github.com//***/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build102471124=/tmp/go-build -gno-record-gcc-switches"
What is the debug output of gocode?
None
What (client-side) flags are you passing into gocode?
potentially -f=json and autocomplete via Go-Plus plugin in Atom?
What editor are using?
Atom
The Problem
Autocompletion works flawlessly except for the case in which the next line starts with // comment or /* comment */. Thus, autocompletion does not work if the code looks like this:
func() {
  somepackage.[no suggestions shown]
  // comment
}

However, this works:
func() {
  somepackage.[no suggestions shown]
}
// comment

Are you synced to the most recent version? I'm not able to reproduce this.

I did clone master and ensured that gocode is not running.. I then ran go install while being inside master. If you are unable to reproduce this, then I will suspect that this is an Atom/Go-Plus issue

@malkhamis go-plus uses mdempsky/gocode by default - just wanted to confirm, have you explicitly installed this fork?

@malkhamis: can you try this in an another editor to confirm that the bug is in Atom? I was not able to reproduce using VSCode.

I cannot reproduce this in Atom either.
2018/12/17 15:41:55 Got autocompletion request for '/Users/zmb3/complete/main.go'
2018/12/17 15:41:55 Cursor at: 48
2018/12/17 15:41:55 -------------------------------------------------------
package main

import "fmt"

func main() {
  fmt.#
  //comment
}
2018/12/17 15:41:55 -------------------------------------------------------
2018/12/17 15:41:55 error in package github.com/zmb3/complete: /Users/zmb3/complete/main.go:6:7:_ not declared by package fmt
2018/12/17 15:41:55 Elapsed duration: 210.343033ms
2018/12/17 15:41:55 Offset: 0
2018/12/17 15:41:55 Number of candidates found: 25
2018/12/17 15:41:55 Candidates are:
2018/12/17 15:41:55   func Errorf(format string, a ...interface{}) error
2018/12/17 15:41:55   func Fprint(w io.Writer, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Fprintf(w io.Writer, format string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Fprintln(w io.Writer, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Fscan(r io.Reader, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Fscanf(r io.Reader, format string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Fscanln(r io.Reader, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Print(a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Printf(format string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Println(a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Scan(a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Scanf(format string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Scanln(a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Sprint(a ...interface{}) string
2018/12/17 15:41:55   func Sprintf(format string, a ...interface{}) string
2018/12/17 15:41:55   func Sprintln(a ...interface{}) string
2018/12/17 15:41:55   func Sscan(str string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Sscanf(str string, format string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   func Sscanln(str string, a ...interface{}) (n int, err error)
2018/12/17 15:41:55   type Formatter interface
2018/12/17 15:41:55   type GoStringer interface
2018/12/17 15:41:55   type ScanState interface
2018/12/17 15:41:55   type Scanner interface
2018/12/17 15:41:55   type State interface
2018/12/17 15:41:55   type Stringer interface
2018/12/17 15:41:55 =======================================================

@stamblerre
I couldn't reproduce this in VSCode either... However, I noticed that vscode would successfully autocomplete albeit I see no output in the terminal while gocode -s --debug is running. I am not a vscode user, so I am unsure if vscode runs multiple autocompletion daemons/process/programs and tries to intelligently autocomplete with best effort?
@zmb3 After trying vscode, it asked me to install stamblerre/gocode and I noticed a binary called gocode-gomod in my GOPATH/bin.. I tried to delete the gocode binary in there and rename the gocode-gomod binary to gocode... I than ran it in terminal in debug mode... I could observe autocompletion requests from Atom, but the same behavior exists (i.e. no autocompletion beyond what is in the current file if the next line is a comment) :(

VSCode runs mdempsky/gocode when users are not in module mode, and it runs this version of gocode, renamed to gocode-gomod to avoid conflicts, in module mode. I'm really unable to reproduce this issue, as you can see from the debug output below.
2018/12/20 13:01:24 Got autocompletion request for '/usr/local/google/home/rstambler/Desktop/gocodetest/main.go'
2018/12/20 13:01:24 Cursor at: 123
2018/12/20 13:01:24 -------------------------------------------------------
package main

import (
	"fmt"

	"rsc.io/quote"
)

func main() {
	fmt.Println("hello")
	quote.Hello()
	quote.Hello()
	quote.#Glass()
}
2018/12/20 13:01:24 -------------------------------------------------------
2018/12/20 13:01:24 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:13:8:_ not declared by package quote
2018/12/20 13:01:24 Elapsed duration: 113.23792ms
2018/12/20 13:01:24 Offset: 0
2018/12/20 13:01:24 Number of candidates found: 4
2018/12/20 13:01:24 Candidates are:
2018/12/20 13:01:24   func Glass() string
2018/12/20 13:01:24   func Go() string
2018/12/20 13:01:24   func Hello() string
2018/12/20 13:01:24   func Opt() string
2018/12/20 13:01:24 =======================================================
2018/12/20 13:01:34 Got autocompletion request for '/usr/local/google/home/rstambler/Desktop/gocodetest/main.go'
2018/12/20 13:01:34 Cursor at: 123
2018/12/20 13:01:34 -------------------------------------------------------
package main

import (
	"fmt"

	"rsc.io/quote"
)

func main() {
	fmt.Println("hello")
	quote.Hello()
	quote.Hello()
	quote.#Glass()
	//hello
}
2018/12/20 13:01:34 -------------------------------------------------------
2018/12/20 13:01:34 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:13:8:_ not declared by package quote
2018/12/20 13:01:34 Elapsed duration: 104.063464ms
2018/12/20 13:01:34 Offset: 0
2018/12/20 13:01:34 Number of candidates found: 4
2018/12/20 13:01:34 Candidates are:
2018/12/20 13:01:34   func Glass() string
2018/12/20 13:01:34   func Go() string
2018/12/20 13:01:34   func Hello() string
2018/12/20 13:01:34   func Opt() string
2018/12/20 13:01:34 =======================================================
2018/12/20 13:02:53 Got autocompletion request for '/usr/local/google/home/rstambler/Desktop/gocodetest/main.go'
2018/12/20 13:02:53 Cursor at: 144
2018/12/20 13:02:53 -------------------------------------------------------
package main

import (
	"fmt"

	"rsc.io/quote"
)

func main() {
	fmt.Println("hello")
	quote.Hello()
	quote.Hello()
	quote.Glass()
	//hello
}

q#
// hello
2018/12/20 13:02:53 -------------------------------------------------------
2018/12/20 13:02:53 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:4:2:"fmt" imported but not used
2018/12/20 13:02:53 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:6:2:"rsc.io/quote" imported but not used
2018/12/20 13:02:53 Elapsed duration: 131.963793ms
2018/12/20 13:02:53 Offset: 0
2018/12/20 13:02:53 Number of candidates found: 1
2018/12/20 13:02:53 Candidates are:
2018/12/20 13:02:53   package quote
2018/12/20 13:02:53 =======================================================
2018/12/20 13:02:54 Got autocompletion request for '/usr/local/google/home/rstambler/Desktop/gocodetest/main.go'
2018/12/20 13:02:54 Cursor at: 149
2018/12/20 13:02:54 -------------------------------------------------------
package main

import (
	"fmt"

	"rsc.io/quote"
)

func main() {
	fmt.Println("hello")
	quote.Hello()
	quote.Hello()
	quote.Glass()
	//hello
}

quote.#
// hello
2018/12/20 13:02:54 -------------------------------------------------------
2018/12/20 13:02:54 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:4:2:"fmt" imported but not used
2018/12/20 13:02:54 error in package github.com/stamblerre/gocodetest: /usr/local/google/home/rstambler/Desktop/gocodetest/main.go:6:2:"rsc.io/quote" imported but not used
2018/12/20 13:02:54 Elapsed duration: 126.898834ms
2018/12/20 13:02:54 Offset: 0
2018/12/20 13:02:54 Number of candidates found: 4
2018/12/20 13:02:54 Candidates are:
2018/12/20 13:02:54   func Glass() string
2018/12/20 13:02:54   func Go() string
2018/12/20 13:02:54   func Hello() string
2018/12/20 13:02:54   func Opt() string
2018/12/20 13:02:54 =======================================================

OK I guess I am the only affected by this for some reason.. I will close this issue then. Thanks a lot for your followup.. really appreciate it

Sorry, I wasn't able to help! I guess my best advice would be to make sure you're synced to head and maybe rebuild it - try go get -u github.com/stamblerre/gocode?