AADB2C90068: The provided application with ID is not valid against this service. Please use an application created via the B2C portal and try again

I have configured a custom policy that just perform signin (I have removed any other custom steps I will use just to make sure the issue is not with policy).
I have followed all instructions for creating IdentityExperienceFramework and ProxyIdentityExperienceFramework. When I run the user flow, it shows the UI screen, posts the request and does not redirect and does not show any errors. Since I enabled application insights, I could see this exception is being thrown.
I am sure I configured the apps as B2C apps and not AD apps, which seems to be a common cause for this issue but this is not the case.
The odd thing is that I have another directory with the same custom policy and with the exactly same configurations and it is working. The only difference is that this directory was created a month ago.
Any thoughts on this issue?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bee63035-00ee-f7fd-97a1-69dae5f808ef
Version Independent ID: a3447928-73cc-0290-6d46-b89234757e3b
Content: Get started with custom policies - Azure AD B2C
Content Source: articles/active-directory-b2c/custom-policy-get-started.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@tiago-alves Thanks for reaching out. To better assist your query, could you please share the document link which you are currently following so that we can involve the appropriate team? If your question is generic and not specific to a document, I would recommend you to please post your query in Microsoft Q&A forum, where we have our engineers and other community members who will be happy to assist.
Once your query is posted in Microsoft Q&A forum, please do share the link here for reference.
And If you have a document which you are following and if you think it needs an update, please let us know about that here.
Thank you.
Thank you

@KranthiPakala-MSFT , thanks for your response.
I am following this document: https://docs.microsoft.com/en-us/azure/active-directory-b2c/custom-policy-get-started
I followed all the steps and now I am facing the issue I reported. Last month I followed the exactly same document and it worked. I am not sure but it looks like an update on Azure AD B2C?

@tiago-alves Thanks for the details. I have added appropriate tags and one of our engineer will review the ask and provide an update as appropriate.

@KranthiPakala-MSFT , thanks.
Just adding some more information. I just found a blog post on the internet from some time ago (when MS released the new app registration portal) and he was facing the same issue and had to use the legacy app registration screen. I don't know if MS added new steps that needs to be performed in order to properly register an application on AD B2C.

@tiago-alves We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.

@tiago-alves - were you able to resolve this? I also am 100% sure I created the app under Azure AD B2C Tenant and NOT Azure AD option, and yet I am in the same space. I have also followed a video tutorial of someone showing it working and verified my setup against theirs, and still getting this same error message.

We are also having the same issue, and are 100% sure that we created the app registration in B2C.

I did get it working if when registering the application I chose "Accounts in any identity provider or organizational directory (for authenticating users with user flows)" instead of "Accounts in this organizational directory only" under the Supported Account Types option. I had tried several other times to create apps and that is the only setting that made a difference. If you look into their help link it sys you MUST select that option to user user flows.

Perfect, Ill have a try at that in the morning, Hopefully thats all it is :)


Compilation errors related to the SqlBinder class and lambda functions passed to execSqlAsync

Description of the Problem:
I am using Drogon's ORM to interact with a MariaDB database. The application involves creating and retrieving Event objects from the database. However, I am facing compilation errors primarily in the EventRepository class, particularly in the insertEvent and getEventById methods. These methods use the execSqlAsync function to execute SQL queries asynchronously.
Errors:
The compilation errors are mostly related to the SqlBinder class and lambda functions passed to execSqlAsync. Here are some of the error messages:
Error related to conversion in SqlBinder.h: cannot convert '(lambda)' to 'uint16_t' without a conversion operator
Errors in FunctionTraits.h about std::remove_reference<int>::type not being a class, namespace, or enumeration.
Incomplete definition errors in SqlBinder and FunctionTraits.

Code Snippets:
Here are the key parts of my code:
void EventRepository::insertEvent(const Playfuse::models::Event& event, std::function<void(bool)> callback) {
std::string query = "INSERT INTO events (...)  FROM events WHERE EventID = ?";
dbClient_->execSqlAsync(query, eventID, ...);
}
I suspect the issue might be related to how I'm passing arguments or lambda functions to execSqlAsync. The application compiles when I comment out the event values in these methods.
Any insights into what might be causing these errors and how to fix them would be greatly appreciated. Thank you in advance for your help!
Best regards,

You made a mistake in the order of parameters in this method. The two callbacks should be the second and third parameters. The parameters bound to the SQL statement should be placed at the end of the parameters.


Refactor confirmOverwrite and downloadSettings

Confirm overwrite I was basically able to copy and paste in exactly from Functions.
downloadAppSettings was a little more tricky.  The refactor requires you to pass in the localFileSettings name and file path since it can vary from extension to extension (specifically App Service vs. SWA and Functions).  Also, only Functions supports encryption/decryption since it relies on the func cli core tools.  The refactor would require the Functions extension to do the decryption before the download and the encryption afterwards.
Not really sure if there's a way around that 🤔

Closing in favor of leveraging the Functions API for this instead.


IsAncestor desc error/contradiction in release notes v0.100.0

This is a bug report for the release notes text, not the software itself.
The v0.100.0 release notes state:

The methods on Page IsDescendant and IsAncestor now returns true for itself, which is obviously the correct behaviour.

The linked ticket #9925 describes the opposite though.
The release notes text is wrong, right? It should say "now returns false for itself"!?
That would match the current versions behavior too.

Fixed.


Crashes on Startup

What are you experiencing an issue with?

[x] Aurora
[ ] Aurora supported game, (Fill in the game)
[ ] Aurora Wrapper
[x] Aurora Profile
[ ] Aurora supported device, (Fill in problematic devices)
[ ] Aurora Scripts
[ ] Aurora Updater
[ ] Aurora Lighting issue
[ ] Other, (Fill in another issue here)

What kind of an issue is it?

[x] Crash
[ ] Performance problems
[ ] Visual bug
[ ] Localization issue
[ ] Enhancement suggestion
[ ] Device issue
[ ] Other, (Fill in another type of issue here)

Expected behavior:
Explain here what should happen.
The app would start up like normal.
Actual behavior:
Explain here what happened instead.
It crashed. (Because I by accidentally hide the desktop profile.)
How often does this problem occur?

This problem occurs... (Every time)
Please describe the steps to reproduce this issue:
The issue can be reproduced by doing following steps:

Hide the desktop profile
Close the app
Open the app

What is the version of Aurora you are experiencing this issue with?

My version is... (v0.6.1.Dev4)
Does the previous version of Aurora have this issue?
The previous version, ((v0.6.1.Dev3), did/did not have this issue.
Please include any logs, screenshots, or files that are related to this issue:

For logs, you can use: http://pastebin.com/
For screenshots, you can use: http://imgur.com/
For files, you can use: http://www.tinyupload.com/ , https://www.dropbox.com/ , or attach them to the issue.

im with the same bug please help

Try deleting %appdata%/Aurora and if that doesn't do anything delete everything from the folder the Aurora exe is in and redownload & install it.

it worked thanks

Fixed with the above commit


Defining or Removing the Update Repository

The System Layer speaks about an 'Update Repository' (https://github.com/International-Data-Spaces-Association/IDS-RAM_4_0/blob/3703bb0b7e6c60d47343bc866197f8a4fa8d245a/documentation/3_Layers_of_the_Reference_Architecture_Model/3_5_System_Layer/3_5_0_System_Layer.md?plain=1#L23), who is responsible for it and defines it? Or shall we dismiss it completely?

Both were removed in the meanwhile. Therefore closing this issue


Multiple-cursors while selecting and pressing option/alt key

It would be great if multiple cursors can be shown while moving the pointer to select a certain region.
To be more specific,On Windows, it can be done by selecting a region while pressing alt key. On macOS, the key is Option.
Meanwhile, a cursor occurred at the end of each selection regions. But,if the cursor position is already at the end of the line, it won't follow the movement of pointer.
This feature on sublime:

(Hope this will clearly illustrate my idea.)

In Jetbrains products, it may look like this:


Feature Request - Utilize the Self Service branding

Another request: we've already got /Users/$USERNAME/Library/Application Support/com.jamfsoftware.selfservice.mac/Documents/Images/brandingimage.png deployed from our Jamf Self Service app with custom branding, would it be possible to utilize that png file instead of having to deploy a second or is it a file resolution/sandboxing limitation to using that one?

@salesloftcb We're testing the following Support Post-install script to leverage brandingimage.png.

Hi @dan-snelson that's awesome! Great workaround as we are unable to directly use the Self Service branding due to App Sandbox. We think App Sandbox is an important security feature and we don't want to disable it.
@salesloftcb As lots of admins are requesting a custom app icon, this is one of the top priorities for an upcoming release!

Hi @salesloftcb in version 2.3 we've added the possibility to modify the icon in alert windows and the about window. Use the key NotificationIcon and restart the app.
Using the Self Service will be hard to implement with App Sandbox. Now that it's open source, perhaps someone knows how to do it :)

Will close this one as we believe this will not be possible without compromising the App Sandbox. Please reopen if proven otherwise


Feature/admin update embargo [OSF-7846]

Purpose
Allows OSF admins to change the embargo date of a registration. This allows our admins to make changes easily if they make a mistake in their registration.
Changes
Added a button to the admin app with this functionality.
Adds info about how to set up the admin app to the readme.
Side effects
None
Ticket
https://openscience.atlassian.net/browse/OSF-7846

@laurenrevere Conflicts


Make it easier to add languages for i18n (and documentation)

In order to add a language to Terrastories, the code has to be modified in a number of places:
application.rb: line 21 (default locale)
routes.rb: line 12 (list of available languages)
welcome/index.html.erb: line 27 (the front page language picker)
config/locales: here, the user has to add the languages as defined in the above places.
We should make it easy enough that the user doesn't have to change any of the code, and document the process of adding new languages.

Taken care of during Ruby by the Bay 2019.


Sample Teams: Add PU, NU, and RU teams

Added:

[Gen 7] PU
[Gen 7] NU
[Gen 7] RU

Also you should name your pull requests more descriptively.

Also you should name your pull requests more descriptively.

Ok! Next update I'll do this!


Fix handling of dates so that we are not comparing the time the pump-…

Fix date-comparison routine so that it captures the date to a variable immediately after pulling the device-clock date, rather than comparing what the clock was set to when the json file was written to disk, to the time it is when the test executes.
This reduces the number of erroneous pump-set calls when CPU load is high.
Also, fix set-device-clocks so that it doesn't try and set the 'clock' on xdrip CGM.

Thanks. Ready to merge to dev, or so we need any specific testing first?

Ready to merge to dev.

This completely failed testing, so I've reverted it after my first attempt at fixing syntax errors revealed even more syntax errors.
Pump clock is more than 55s off: attempting to reset it
Waiting for ntpd to synchronize... OK!
Setting pump time to Wed Feb 14 19:40:45 PST 2018
mmeowlink.exceptions.CommsException: No acknowledgement from pump on wakeup. Is it out of range or is the battery too low?
/usr/local/bin/oref0-pump-loop: line 167:<PHONE_NUMBER>=1518666072: command not found
sed: -e expression #1, char 7: unknown option to `s'
date: option requires an argument -- 'd'
Try 'date --help' for more information.
/usr/local/bin/oref0-pump-loop: line 169: let: DTG_DIFFERENCE=1518666018-: syntax error: operand expected (error token is "-")
Error: pump clock refresh error / mismatch

Pump clock is more than 55s off: attempting to reset it
Waiting for ntpd to synchronize... OK!
Setting pump time to Wed Feb 14 19:48:12 PST 2018
mmeowlink.exceptions.CommsException: No acknowledgement from pump on wakeup. Is it out of range or is the battery too low?
sed: -e expression #1, char 7: unknown option to `s'
date: option requires an argument -- 'd'
Try 'date --help' for more information.
/usr/local/bin/oref0-pump-loop: line 171: [:<PHONE_NUMBER>-: integer expression expected
Pump clock is now within 90s of current time. Proceeding.

Those errors are caused by invalid data in monitor/clock-zoned.json (eg, if the pump was out of range & something other than an epoch DTG - or nothing at all - is in that file).
That resulted in no value being presented after the '-' sign in the subtraction operation, and thus a syntax error.
I've updated the code to detect & handle that.
Will let you know when I've confirmed it catches the errors & prevents them.

Here's the contents of those files at the time.  They were way stale, but not invalid.
root@edison-eb5 ~/myopenaps # cat monitor/clock.json
"2018-02-14T17:43:11"root@edison-eb5 ~/myopenaps # cat monitor/clock-zoned.json
"2018-02-14T17:43:11-08:00"root@edison-eb5 ~/myopenaps # ll monitor/clock*
-rw-r--r-- 1 root root 21 Feb 14 17:43 monitor/clock.json
-rw-r--r-- 1 root root 27 Feb 14 17:43 monitor/clock-zoned.json

Got it.
Will test that too...
The actual syntax error was the missing second operand in the subtraction problem, which would result from a falure of something in the $(cat monitor/clock-zoned.json ...) statement...
Under your original code (I copied & pasted the file-extraction code, so it should be the same cat/sed/etc statement), that would just fail the entire section (due to the && s) and I don't know how BC would handle it, under mine... not so much, plus a syntax error...


[FLINK-26137] Create webhook REST api test

Adds unit tests to validate the webhook rest endpoint and make sure it returns the expected responses, status codes etc.
The brief change log

Introduces AdmissionHandlerTest to adds the test cases testHandleIllegalRequest, testHandleValidateRequestWithoutContent and testHandleValidateRequestWithAdmissionReview for the validation of the webhook rest endpoint.

@gyfora @wangyang0918 @tweise, could you please help to review?

@gyfora @wangyang0918 @tweise , could you please help to review this pull request?

@gyfora, thanks for detailed review. I have addressed the above comments. Please help to take a look.


Adding moduler Content

Motivation
Which issue does this fix? Fixes #issue number
If no issue exists, what is the fix or new feature? Were there any reasons to fix/implement things that are not obvious?
Checklist

[ ] Code follows coding conventions held in this repo
[ ] Automated tests have been added
[ ] Tests are passing
[ ] Docs have been updated (if applicable)
[ ] Temporary settings (e.g. variables used during development and testing) have been reverted to defaults

How to test
If manual testing is required, what are the steps?

Hello @Kunjbihari1509.
could you describe your use case? It would be more beneficial to decide about the feature definition.

Hi!
This issue has gone quiet. 👻
It’s been a while since the last update here.
If we missed anything on this issue or if you want to keep it open, please reply here.
Thanks for being a part of the Kentico community!


Updated README to document used CSS custom properties.

It's required to set these properties in order for the editor controls to correctly render.
Update: Fixes #5 (apparently 😅)

Yes, thank you! I'm going to merge this, but will probably introduce a set of new properties to replace them. The current names are all from django.contrib.admin and are a bit too likely to clash.


Mention node.js requirement in readme

I tried building rust-analyzer according to the instructions, but it failed with a very non-descriptive error:

will run: npm ci
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

It took me a while to figure out I had an outdated node version installed, which didn't support npm ci. I think mentioning the requirement explicitly might prevent others from running into the same issue.

LGTM. I ran into this last week.

I am still not convinced that npm ci is the right command for cargo install-code, but :shrug:
bors r+


How to Update

Hi,
I downloaded the mac desktop version - how can I update to the latest version without losing my conversation history?

You can export the data by clicking the "Import / Export" button on the sidebar. When you download a new version, you can import the data in the new version.

You shouldn't lose your conversation history when you update though, because it's saved in ~/Application Support/better-chatgpt and the app update should not even touch the folder.


Successfully build the app on iOS

The app should:

Start

@nicholemnl This is done, right?

Hi Ken it would be in on this branch feature/ios-129
However, just to note, this can't be merged to develop yet since I have to remove some of the feature that was causing some issue in IOS.

Tested on iOS build version 0.4.0 (3)
Verified that the iOS build can be installed, launched, and set up.
Test devices:

iPhone 11 (iOS 15.5)
iPhone SE (iOS 15.4)

Works as expected


I know To little

Hello!
Last year I went into this project. I bought all the BUM. Yesterday I did some progress and now I am stuck.
So I know to little on the programming side of RPI HA.
What I did...
SOlder the EZO circuits on carrier board, connected the thing per your scheme.
Installed HA operating system 64bit via this tutorial  https://www.home-assistant.io/installation/raspberrypi
Copied all the project files as the instructions were.
1. Now the first problem is that when I check configuration it says

2. Then the lovelace looks like this

3. I dont know coding but somehow it doesn't read the sensor data from the atlas_scientific custom components folder...
Should I do something with this since it look like this? (example.yaml filename is ok??)

4. This is how the connection looks like:

I sure am missing something basic but I just dont know.  Can someone please help me, since I bought all this materials and now I dont know how to make them work.
Thank you
Boris

Did you install HA as a container or via python venv? Either way, the custom component seems to be in the right place.
Check if RPi can read from your sensor (https://files.atlas-scientific.com/pi_sample_code.pdf). Then if all that is correct, check your HA logs and post what you are seeing.
Adding this in your configuration.yaml might help you debug further:
logger:
  default: info
  logs:
    custom_components.atlas_scientific.sensor: debug

Hello virgilm!
Thank you for the quick response.
Installed HA operating system 64bit via this tutorial https://www.home-assistant.io/installation/raspberrypi
So no raspberry pi OS
The atlas scientific pdf you sent i cannot get to the config (sudo raspi-config command) since it is a HA OS.
my configuration.yaml is like this so the debug for atlas scientific sensor is there.
Any suggestions or did I do everything wrong...
Configure a default setup of Home Assistant (frontend, api, etc)
default_config:
Text to speech
tts:
- platform: google_translate
group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml
homeassistant:
name: RaspiPool
Location required to calculate the time the sun rises and sets
latitude: 46.4453269259808
longitude: 15.857537091648812
unit_system: metric
customize: !include customize.yaml
packages: !include_dir_named packages
default_config:
Disable Discover some devices automatically
discovery:
tts:

platform: google_translate
logger:
default: critical
logs:
homeassistant.components.sensor.atlas_scientific: debug
homeassistant.components.switch.command_line: debug

group: !include groups.yaml
Disable automation
automation: !include automations.yaml
script: !include scripts.yaml
lovelace:
mode: yaml

Please format your file...
Well, that's because HAAS or HA OS or whatever is called nowadays is a joke! (at least on RPi). You can try this to make sure serial is configured. Send the logs (from your config/.home-assistant.log). Good luck!

Sorry for the format I copied and pasted it and that is what turned out :(
I edited the upper post.
So you suggest to install raspberry pi OS and then the HA container?

here is the log.
home-assistant.log

Hello again
This I dont find in the addon store on my HA OS. Maybe because it is not HAAS anymore?
Confused...
https://github.com/adamoutler/HassOSConfigurator

Can't clear that confusion for you, HA made it as hard as possible to figure it out... I'm not suggesting anything about HA, except that is evolving in the wrong direction (at least for this project).
You can fix your problem at hand by adding the following line to your custom_components/atlas_scientific/manifest.json
  "version": "1.0.0",. Another breaking change in HA - they don't care about backward compatibility one bit.
That will get you unstuck at least. I have a fork here: https://github.com/virgilm/raspipool, and I will try to bring it up to date, but the interest in this project seems to have waned...

Hmm
tried the line nothing happened
I overwrote your fork put in the longitude and lattitude, pushbullet and weather APIs and now this happens

Stuck here.
I want to write a tutorial for all of this step by step on this date and I hope its not that difficult... anyway... :)

Don't get my fork... it has some components that need more support, and I'm not sure I'm ready to support that fork :)
Just change your manifest file to read:
{
  "version": "1.0.0",
  "domain": "atlas_scientific",
  "name": "Atlas Scientific",
  "documentation": "https://github.com/home-assistant/example-custom-config/tree/master/custom_components/example_sensor/",
  "dependencies": [],
  "codeowners": [],
  "requirements": ["pyserial", "smbus-cffi==0.5.1"]
}

HMM i did that and still problems... the same

Does maybe anybody have the SD-card Image for raspberry that just works? Because I cannot get that to work.
PLEASE...!

I put up my latest working fork at https://github.com/virgilm/raspipool. Your sensors would have to be in I2C mode and the installation is not trivial. Good luck!

OK I will try this hope it works! How do I set sensors in I2C mode?
Thank you


remove logbook

resolves #8027
Problem
We don't use logbook anymore.
Solution
Remove it and all references to it.
Checklist

[ ] I have read the contributing guide and understand what's expected of me
[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
[ ] This PR includes type annotations for new and modified functions

The only thing that was modified in the artifacts dir was a comment so added the artifact_minor_upgrade label.


possible to fix this?

https://github.com/IceFireDB/redhub/issues/8
it shld be an opened issue

any quick fix to this issue for the moment? as "address already in use" this way is slowing down testing. need to wait. how to quickly ensure the port is ready for reuse?

Hi, in the latest commit, added whether SO_REUSEPORT socket option is set.
You can set reusePort=true at startup.

ok thx


Toggle Button import issue in material-ui/lab

Module not found: Can't resolve '@material-ui/lab/ToggleButton' in 'D:\SelfStudy\web-dev\React\filmzilla\src\comps'

in latest version, this error is coming on browser, when i installed @46 , it worked

But i need to know whats issue in latest version @

Please provide a codensadbox showing the issue. It's not even clear which version you are using. Are you on v4 or v5?

Please provide a codensadbox showing the issue. It's not even clear which version you are using. Are you on v4 or v5?
hey thanks for replying, actually m using vscode, and for now its working for me, if some error comes in future, i ll surely upload on sandbox and reply to this comment! thanks again


Allow query params for scrape endpoints.

Getting different scrape contents based on query parameters can be a useful addition.
The discussion started with #674. As stated there, I would like to avoid proxying by query parameters but only use them for actual modifications. But the line gets blurry quite fast.
As the issue of the PR will be resolved by relabeling, this is not too urgent for now.

With #786 this becomes reality. We need to define some rules regarding parameter names.
I'd propose:

_.+ for exporting applications
__.+ for anything standing between an exporting application and Prometheus
[^_]+ for the Prometheus handler

I think encouraging _ for applications we don't control would be tricky, and suggest this logic be put the other way around. I'd also expect that anyone passing things to endpoints is going to either be sufficiently custom that there's nothing you'd want to pass to the Prometheus handler, or you're only talking to the prometheus handler.

I'm not really willing to do prefixing for fundamental parameters of the actual application in favor of some potential custom setup. Especially because it doesn't guarantee anything if we assume the user may use anything.
We can also go for 0 restrictions, given the low chance of actual collisions, and expect people to do some renaming in their reverse proxies if required.

Query parameters implemented via #786.
TODO: Expose query parameters for relabeling.

#934 added relabelling.


[GraphQL Transform] Filtering with multiple many-to-many relationships

** Which Category is your question related to? **
GraphQL Transform
** What AWS Services are you utilizing? **
Appsync, Amplify
** Provide additional details e.g. code snippets **
If I have a schema like this
type Post @model {
  id: ID!
  title: String!
  editors: [PostEditor] @connection(name: "PostEditors")
  categories: [PostCatoegory] @connection(name: "PostCategories")
}

type PostEditor {
  id: ID!
  post: Post! @connection(name: "PostEditors")
  editor: User! @connection(name: "UserEditors")
}
type User @model {
  id: ID!
  username: String!
  posts: [PostEditor] @connection(name: "UserEditors")
}

type PostCatoegory {
  id: ID!
  post: Post! @connection(name: "PostCategories")
  category: Category! @connection(name: "CategoryType")
}
type Category @model {
  id: ID!
  categoryName: String!
  posts: [PostCatoegory] @connection(name: "CategoryType")
}

This is how I can get the posts' titles that are posted by the user "ibrahim"
    query {
    	listUsers(
        filter:{
          username:{
            eq: "ibrahim"
          }
        })
      {
        items{
          posts{
            items{
              post{
                title
              }
            }
          }
        }
      }
    }

And I will do a similar thing to get the posts which have a specific categoryName, but my question is how can I get the posts that are posted by let's say "ibrahim" and have a categoryName "art" ?
I have spent a lot of time trying to solve this issue and will appreciate your help

Hey @ialmoqren thanks for the question. We have an RFC coming out the details some new features around custom indexes that will give you more control over how the tables created by @model are configured. In general, to find the posts that are posted by "ibrahim" and that have a categoryName "art" will require an index on the "Post" table that lies on top of these two attributes.
For example, if I have a table "Post" with a GSI with a hash key of "userId" and a sort key of "categoryName" then I can do a DynamoDB query operation against that table to find all the posts with that "userId" and "categoryName". I will tag this issue when the custom indexes RFC is released so we can discuss if it will solve your use case.

Thanks @mikeparisstuff a lot for your response. I obviously need to learn more about DynamoDB's GSIs, but please allow me to ask you this, should those indexes be created through aws console (website) or can I declare them directly from my schema.graphql.
I couldn't find resources about how to create DynamoDB's GSIs when using Amplify, I would really appreciate it if you can suggest me any so I can manage to solve my issue.
I can see that it's possible to specify the keyField and sortField on a connection, but in my case it's two connections not one. If that's what you mean.
Many thanks for your time, I understand how busy you are.

@ialmoqren We are going to add support for custom indexes via the schema.graphql. We have an RFC open here https://github.com/aws-amplify/amplify-cli/issues/1062 and would love to hear your feedback on the design.

COPIED FROM #1062

@mikeparisstuff Thanks for your time and effort, but I believe none of those changes would tackle this issue,
Please let me re-explain my issue,
Let's say I have a "Post" type as following:
type Post @model {
  id: ID!
  title: String!
  editors: [PostEditor] @connection(name: "PostEditors")
  categories: [PostCatoegory] @connection(name: "PostCategories")
  likes: [PostLike] @connection(name: "PostLikes")
}

The "editors", "categories" and "likes" have MANY-TO-MANY relationships with "Writer", "Category" and "Reader" types respectively.
I can easily get:

The posts someone wrote
The posts in one category
The posts someone liked

with something like this:
    query {
    	listWriters(
        filter:{
          name:{
            eq: "X"
          }
        })
      {
        items{
          posts{
            items{
              post{
                title
              }
            }
          }
        }
      }
    }

But, my issue is how can I combine these filters, I need to get the posts that are written by the writer X, and in the category Y and are liked by the reader Z.
I believe this use case is very common in many apps and websites, think about things like marketplaces which have many filters based on many criteria.
My example has three filters but you can think of any number of filters someone may have.
It's possible (and I hope) that the issue is much simpler than how I am thinking about it.
I really appreciate your effort.

@ialmoqren at this moment, I don't believe you are able to filter many-to-many relationships. I'm running into the same issue where we have a many-to-many relationship:
type User
  @model
{
  id: ID!
  projects: [ProjectUser] @connection(name: "UserMembers", sortField: "createdAt")
  createdAt: AWSDateTime
}

A ProjectUser model to join the two:
type ProjectUser @model(queries: null) {
  id: ID!
  project: Project! @connection(name: "ProjectMembers")
  member: User! @connection(name: "UserMembers")
  createdAt: AWSDateTime
}

type Project
  @model
{
  id: ID!
  title: String!
  members: [ProjectUser] @connection(name: "ProjectMembers")
  createdAt: AWSDateTime
}

And when it gets created, I notice that the input filter looks like this:
input ModelProjectUserFilterInput {
	id: ModelIDFilterInput
	createdAt: ModelStringFilterInput
	and: [ModelProjectUserFilterInput]
	or: [ModelProjectUserFilterInput]
	not: ModelProjectUserFilterInput
}

Which is severely limited in filtering the information necessary. This obviously is a huge limitation with the many-to-many relationship and needs to be addressed somehow.

@ialmoqren We launched support for custom indexes today. You can find docs for the same out here - https://aws-amplify.github.io/docs/cli/graphql#key
Please let us know if you're still not able to solve your problem through this solution and we'll re-open this issue for you.

@kaustavghosh06 I don't believe custom indexes will solve this problem.
I'm facing a related and very common problem, I need to filter a many-to-many connection (which in my case is a list of User's friends) by fields defined on the user:
type User @model @searchable {
	id: ID!
	userName: String!
    firstName: String!
	lastName: String
	email: String
	phoneNumber: String
	handle: String
	isInitialized: Boolean!
	updatedAt: String
	createdAt: String
	friends: [UserFriend] @connection(name: "UserFriends")
	friendsOf: [UserFriend] @connection(name: "FriendUsers")
}
type UserFriend @model @searchable {
  id: ID!
  user: User! @connection(name: "UserFriends")
  friend: User! @connection(name: "FriendUsers")
}

This doesn't allow me to filter on any fields defined on UserFriends.User, through the @connection attribute. The generated graphql input type is:
export type SearchableUserFriendFilterInput = {
  id?: SearchableIDFilterInput | null,
  and?: Array< SearchableUserFriendFilterInput | null > | null,
  or?: Array< SearchableUserFriendFilterInput | null > | null,
  not?: SearchableUserFriendFilterInput | null,
};

Making any kind of nested filtering impossible for many-to-many relationships.

https://github.com/aws-amplify/amplify-cli/issues/2598
A thread I started a couple of weeks ago while starting to use Appsync and I see no way to do filtering over many-to-many relations.

Re-opening per https://github.com/aws-amplify/amplify-cli/issues/2598#issuecomment-557847223. Please comment on #2598 if thats the accurate use case

I have another use case for this exact issue. #3003 Has there been any progress on filtering many-to-many connections?

I agree that filtering over many-to-many relations is not possible at the moment even though it has a lot of use cases.
For example, in my case, I have a many-to-many friendship relation between users. Each user has a status field. Currently, there is no straightforward way to query the friends of a specific user with status "active". Is this kind of functionality in the queue?

I agree with @avlonder the use of join table in the amplify docs goes against the single table best practice in DynamoDB docs.

Hey !
Is someone have found a solution for this problem?
I have the same issue. Even with searchable data (with elastic instance) is not possible.

I would suggest that everyone takes a look at DynamoDB documentation, more specifically, single table design. The database was not made for many-to-many connections, it is a misuse and the Amplify docs are misleading users into thinking it is was made to work like that. Alex DeBrie and AWS have made some good videos (that are on youtube) which explain how to best design for DynamoDB. The Amplify team need to re-write their docs and stop pushing relation database design whilst using a NoSQL database.


Masterful Whirl doesn't seem to be working

When I have it selected, the reaction dialog pops up but the text in the button looks like code. I click the button, do the attack, and it seems like only the regular weapon damage is being done.
Player.log
whirl.zip

You are using <IP_ADDRESS>. I only support the latest version of the mod.


[rule-detail] Add rule detail page with examples of configuration

Add configuration example to copy-paste and try :) ↓

Config now meets coding standards :)

Next iteration :)


Model checkpoint issue: serialization error for tf.string

TF version: 1.8
Installation: tensorflow/tensorflow:latest-gpu
OS details:
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.4 LTS
Release:	16.04
Codename:	xenial
AWS instance: p2.xlarge
GPU details:
[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation:<PHONE_NUMBER>9116902206
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit:<PHONE_NUMBER>1
locality {
  bus_id: 1
  links {
  }
}
incarnation:<PHONE_NUMBER>9777279899
physical_device_desc: "device: 0, name: Tesla K80, pci bus id: 0000:00:1e.0, compute capability: 3.7"
]

Custom code:
use_model = hub.Module("http://tfhub.dev/google/universal-sentence-encoder/1", trainable=True);
sess.run(tf.global_variables_initializer());
sess.run(tf.tables_initializer());

def USEEmbedding(x):
    return use_model(tf.squeeze(tf.cast(x, tf.string)),
                      signature="default", as_dict=True)["default"]

input_text = layers.Input(shape=(1,), dtype=tf.string)
embedding = layers.Lambda(USEEmbedding, output_shape=(512,))(input_text)
dense = layers.Dense(1024, activation='relu')(embedding)
bnorm = layers.BatchNormalization()(dense)
pred = layers.Dense(2000, activation='softmax')(bnorm)

model = Model(inputs=[input_text], outputs=pred)

model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()

callbacks = [keras.callbacks.EarlyStopping(monitor='val_acc',
                                               min_delta=1e-3,
                                               patience=8,
                                               verbose=0,
                                               mode='auto'),
             keras.callbacks.ModelCheckpoint('../models/best-weights.h5',
                                                 monitor='val_acc',
                                                 verbose=1,
                                                 save_best_only=True,
                                                 mode='auto'),
             keras.callbacks.TensorBoard(log_dir='../tb-logs', histogram_freq=0,
                                         write_graph=True, write_images=False)]

train_text = [' '.join(t.split()[0:20]) for t in train_x.sentences.tolist()]
train_text = np.array(train_text, dtype=object)[:, np.newaxis]
valid_text = [' '.join(t.split()[0:20]) for t in valid_x.sentences.tolist()]
valid_text = np.array(valid_text, dtype=object)[:, np.newaxis]

model.fit(train_text , train['labels'].tolist(),
          validation_data=(valid_text, valid['labels'].tolist()),
          epochs=100, batch_size=256,
          callbacks=callbacks, shuffle=True)

Error:

TypeError                                 Traceback (most recent call last)
<ipython-input-81-e546b70750dc> in <module>()
      2           validation_data=(valid_text, valid['labels'].tolist()),
      3           epochs=100, batch_size=256,
----> 4           callbacks=callbacks, shuffle=True)

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
   1703                               initial_epoch=initial_epoch,
   1704                               steps_per_epoch=steps_per_epoch,
-> 1705                               validation_steps=validation_steps)
   1706
   1707     def evaluate(self, x=None, y=None,

/usr/local/lib/python2.7/dist-packages/keras/engine/training.pyc in _fit_loop(self, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)
   1254                             for l, o in zip(out_labels, val_outs):
   1255                                 epoch_logs['val_' + l] = o
-> 1256             callbacks.on_epoch_end(epoch, epoch_logs)
   1257             if callback_model.stop_training:
   1258                 break

/usr/local/lib/python2.7/dist-packages/keras/callbacks.pyc in on_epoch_end(self, epoch, logs)
     75         logs = logs or {}
     76         for callback in self.callbacks:
---> 77             callback.on_epoch_end(epoch, logs)
     78
     79     def on_batch_begin(self, batch, logs=None):

/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/callbacks.pyc in on_epoch_end(self, epoch, logs)
    466               self.model.save_weights(filepath, overwrite=True)
    467             else:
--> 468               self.model.save(filepath, overwrite=True)
    469           else:
    470             if self.verbose > 0:

/usr/local/lib/python2.7/dist-packages/keras/engine/topology.pyc in save(self, filepath, overwrite, include_optimizer)
   2589         """
   2590         from ..models import save_model
-> 2591         save_model(self, filepath, overwrite, include_optimizer)
   2592
   2593     def save_weights(self, filepath, overwrite=True):

/usr/local/lib/python2.7/dist-packages/keras/models.pyc in save_model(model, filepath, overwrite, include_optimizer)
    125             'class_name': model.__class__.__name__,
    126             'config': model.get_config()
--> 127         }, default=get_json_type).encode('utf8')
    128
    129         model_weights_group = f.create_group('model_weights')

/usr/lib/python2.7/json/__init__.pyc in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, encoding, default, sort_keys, **kw)
    249         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    250         separators=separators, encoding=encoding, default=default,
--> 251         sort_keys=sort_keys, **kw).encode(obj)
    252
    253

/usr/lib/python2.7/json/encoder.pyc in encode(self, o)
    205         # exceptions aren't as detailed.  The list call should be roughly
    206         # equivalent to the PySequence_Fast that ''.join() would do.
--> 207         chunks = self.iterencode(o, _one_shot=True)
    208         if not isinstance(chunks, (list, tuple)):
    209             chunks = list(chunks)

/usr/lib/python2.7/json/encoder.pyc in iterencode(self, o, _one_shot)
    268                 self.key_separator, self.item_separator, self.sort_keys,
    269                 self.skipkeys, _one_shot)
--> 270         return _iterencode(o, 0)
    271
    272 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

/usr/local/lib/python2.7/dist-packages/keras/models.pyc in get_json_type(obj)
    102             return obj.__name__
    103
--> 104         raise TypeError('Not JSON Serializable:', obj)
    105
    106     from . import __version__ as keras_version

TypeError: ('Not JSON Serializable:', tf.string)

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

@fchollet looks like a keras issue

I had the same issue and replacing
input_text = layers.Input(shape=(1,), dtype=tf.string)

by
input_text = layers.Input(shape=(1,), dtype="string")

solved it for me.

Nagging Assignee @fchollet: It has been 15 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 30 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 45 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 60 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 75 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 90 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 105 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 120 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 135 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 165 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 179 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @fchollet: It has been 195 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Possibly related: https://github.com/keras-team/keras/issues/12473

This is fixed with latest tf-nightly build. Thanks!


heartfelt gratitude

hey! brother, thank u very much !!!
I can't find your contact information, so this is the only way I can talk to you.
this methods here provided by you has solved my question.
thank u very much!!!

You're welcome.


Playback

Not working

Thanks a lot Sir.
On Sun, 19 Dec 2021, 9:00 pm Jake Wharton, @.***> wrote:

Closed #1667 https://github.com/JakeWharton/butterknife/issues/1667.
—
Reply to this email directly, view it on GitHub
https://github.com/JakeWharton/butterknife/issues/1667#event-5789275121,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AUDVDRRQJ7YCPV5P3NH7AVTURX6SDANCNFSM5KL6AZSQ
.
You are receiving this because you authored the thread.Message ID:
@.***>


WIP fix(components): remove z-index usage

Description
Remove z-index usage in the modal, pagination, popover and table components
Scope

[ ] Patch: Bug Fix
[ ] Minor: New Feature
[ ] Major: Breaking Change

should we add a screener.io scenario with an open select dropdown in a modal? or something similar, to have a visual test case for it?

Can we also remove it from create.js and styles/types.js

Can we also remove it from create.js and styles/types.js

It will be a breaking change. We can remove it on the next major bump.


Exporting .carto smoke test #99

smoke test task #99 asks to Export a Full Map. The dialogue that comes up, has the "Ok, Export" option greyed out. If I click anywhere on the page other than the greyed out button, I get the option to download.
In short, the export is possible, but the dialogue seems to be broken.

Tried it again and if I click on the greyed out button it downloads but the "Ok, Export" button is always greyed out.

I've tried in prod with several maps and it all seems good :(


How to load custom models that contains multiple files

I tried multiple ways, particularly for RU EN Model 2019 that has 4 files names model1.pt, model2.pt and more. I checked the code and do not see how to load multiple files. Could you help and add to the manual as i guess many people have the same problem.
 from fairseq.models.transformer import TransformerModel
 zh2en = TransformerModel.from_pretrained(
  '/path/to/checkpoints',
  checkpoint_file='checkpoint_1.pt  checkpoint_2.pt checkpoint_3.pt ',
  data_name_or_path='data-bin/wmt17_zh_en_full',
  bpe='subword_nmt',
  bpe_codes='data-bin/wmt17_zh_en_full/zh.code'
)
zh2en.translate('你好 世界')
# 'Hello World'

Thanks, I just updated the README with more detailed instructions, but you need to separate the model checkpoints with :, like this:
 zh2en = TransformerModel.from_pretrained(
  '/path/to/checkpoints',
  checkpoint_file='checkpoint_1.pt:checkpoint_2.pt:checkpoint_3.pt',
  data_name_or_path='data-bin/wmt17_zh_en_full',
  bpe='subword_nmt',
  bpe_codes='data-bin/wmt17_zh_en_full/zh.code'
)


APP: restart npm server when you've changed packages code

currently editing totum and other submodules in the packages directory requires a manual restart to apply.  watch the packages folder for local changes and restart node when they occur.

assigned to @abeersaqib as his work should address this.  please link this ticket to the relevant issues


can not upload more images

i just upload adout 230 mb about 100 images。but it stuck.

I have experienced the same issue. Have not looked much into it, but found a quick workaround by uploading 60 images at a time.
I set it up locally with pip and am running it from windows terminal:
pip install label-studio
label-studio
Then I navigate to localhost:8080 in browser. I have a hunch that running it in this way has its limitations 😅

upload images accroding it's api maybe works.i didn't try it.

It's hard for us to tell what's wrong here without more information about your setup, but splitting your uploads into batches as @haraldbo suggests seems pretty reasonable to me. If you'd like to reopen, please provide more details about your system, how you're running label studio, what error behavior exactly you're running in to, and so on.


Integration with elasticsearch v5

Hi
I know that the new version v5 of elasticsearch has just being released, but do you know when it will be available the integration of elastalert with the new version? At the moment I am getting:

INFO:elastalert:Starting up
WARNING:elasticsearch:POST http://elasticsearch:9200/elastalert_status/elastalert/_search?size=1000 [status:400 request:0.013s]
WARNING:elasticsearch:POST http://elasticsearch:9200/elastalert_status/elastalert_status/_search?_source_include=endtime%2Crule_name&size=1 [status:400 request:0.002s]
ERROR:root:Error querying for last run: TransportError(400, {u'line': 1, u'root_cause': [{u'reason': u'Unknown key for a START_OBJECT in [filter].', u'type': u'parsing_exception', u'line': 1, u'col': 12}], u'type': u'parsing_exception', u'reason': u'Unknown key for a START_OBJECT in [filter].', u'col': 12})
WARNING:elasticsearch:POST http://elasticsearch:9200/elastalert_status/elastalert_error?op_type=create [status:400 request:0.003s]
ERROR:root:Error writing alert info to elasticsearch: TransportError(400, {u'root_cause': [{u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;', u'type': u'action_request_validation_exception'}], u'type': u'action_request_validation_exception', u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;'})
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1033, in writeback
doc_type=doc_type, body=body)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/init.py", line 227, in create
return self.index(index, doc_type, body, id=id, params=params, op_type='create')
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
return func(args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/init.py", line 257, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 301, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 72, in perform_request
self._raise_error(response.status_code, raw_data)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 102, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: TransportError(400, {u'root_cause': [{u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;', u'type': u'action_request_validation_exception'}], u'type': u'action_request_validation_exception', u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;'})
WARNING:elasticsearch:GET http://elasticsearch:9200/payment-/_search?_source_include=eventtime%2C%2A%2CuserId&ignore_unavailable=true&size=10000 [status:400 request:0.003s]
ERROR:root:Error running query: TransportError(400, {u'line': 1, u'root_cause': [{u'reason': u'no [query] registered for [filtered]', u'type': u'parsing_exception', u'line': 1, u'col': 67}], u'type': u'parsing_exception', u'reason': u'no [query] registered for [filtered]', u'col': 67})
WARNING:elasticsearch:POST http://elasticsearch:9200/elastalert_status/elastalert_error?op_type=create [status:400 request:0.003s]
ERROR:root:Error writing alert info to elasticsearch: TransportError(400, {u'root_cause': [{u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;', u'type': u'action_request_validation_exception'}], u'type': u'action_request_validation_exception', u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;'})
Traceback (most recent call last):
File "/opt/elastalert/elastalert/elastalert.py", line 1033, in writeback
doc_type=doc_type, body=body)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/init.py", line 227, in create
return self.index(index, doc_type, body, id=id, params=params, op_type='create')
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
return func(*args, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/init.py", line 257, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 301, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 72, in perform_request
self._raise_error(response.status_code, raw_data)
File "/usr/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 102, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: TransportError(400, {u'root_cause': [{u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;', u'type': u'action_request_validation_exception'}], u'type': u'action_request_validation_exception', u'reason': u'Validation Failed: 1: an id must be provided if version type or value are set;'})
INFO:elastalert:Ran User1111MakePayment from 2016-11-02 08:20 UTC to 2016-11-02 13:20 UTC: 0 query hits, 0 matches, 0 alerts sent

Thanks for this great tool!!!!

ES 5 has been around for a while already, its the GA version that just came out, any ETA on the support would be lovely :+1:

+1

+1

+1
On Sat, Nov 5, 2016 at 5:14 AM -0700, "lrolsen" <notifications<EMAIL_ADDRESS>wrote:
+1

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/Yelp/elastalert/issues/790#issuecomment-258607858, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AH7j4V-D_DPsJ-KmIqww-sOMgZ5FaPG-ks5q7HMggaJpZM4KnOvN.

@watollop says me to put a +1

+1

+1

+1

+1 Duplicate of #510

+1

+1

Is it known what is broken and how can we help ?

IRC pointed me to https://github.com/Yelp/elastalert/blob/master/elastalert/elastalert.py#L159

+1

Is it known what is broken and how can we help ?
as i known, it is because "filter" is not available any more in ES 5.0.
There are many "filter" query in EA which will get the 400 HTTP response for the incorrect query clause.

+1

+1

It looks like all needed is to rewrite this query:
https://github.com/Yelp/elastalert/blob/master/elastalert/elastalert.py#L159
(thanks @bkeifer )
Documentation says that change should be simple: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/query-dsl-filtered-query.html
I'll try to find some time, my python is a bit rusty :)

My reading of that doc suggests that the change should be:
diff --git a/elastalert/elastalert.py b/elastalert/elastalert.py
index 2cbd553..a13e739 100644
--- a/elastalert/elastalert.py
+++ b/elastalert/elastalert.py
@@ -156,7 +156,7 @@ class ElastAlerter():
         if starttime and endtime:
             es_filters['filter']['bool']['must'].insert(0, {'range': {timestamp_field: {'gt': starttime,
                                                                                'lte': endtime}}})
-        query = {'query': {'filtered': es_filters}}
+        query = {'query': {'bool': es_filters}}
         if sort:
             query['sort'] = [{timestamp_field: {'order': 'desc' if desc else 'asc'}}]
         return query

I'll see if I can make it work in our ES5 environment

@timwsuqld: plus if statement to detect ES version

@stumyp Good point. Any idea if we already detect ES version anywhere?

@timwsuqld : So far I saw only this: https://github.com/Yelp/elastalert/commit/63584350031599a59a90816c85fb2de7ed5a1bd0
Not really a version detection :)

I'm also not sure if the elasticsearch library (https://pypi.python.org/pypi/elasticsearch/5.0.1) is backwards compatible. My understanding is that it should be, so we can use version 5 of the library with older ES clusters.

From what they say on the link you gave: major version of library must match ES version.
Means, elastalert should split versions too or just warn users for incompatibility ?

I believe this patch must also be applied, because id must be explicitly set to None.
diff --git a/elastalert/elastalert.py b/elastalert/elastalert.py
index 2cbd553..fd49223 100644
--- a/elastalert/elastalert.py
+++ b/elastalert/elastalert.py
@@ -841,7 +841,8 @@ class ElastAlerter():

         res = es.create(index='kibana-int',
                         doc_type='temp',
-                        body=db_body)
+                        body=db_body,
+                        id=None)

         # Return dashboard URL
         kibana_url = rule.get('kibana_url')
@@ -1015,7 +1016,7 @@ class ElastAlerter():
         if self.writeback_es:
             try:
                 res = self.writeback_es.create(index=self.writeback_index,
-                                               doc_type=doc_type, body=body)
+                                               doc_type=doc_type, body=body, id=None)
                 return res
             except ElasticsearchException as e:
                 logging.exception("Error writing alert info to Elasticsearch: %s" % (e))

For what it's worth, I appear to have no problems using the elasticsearch 2.4.0 library, and the index creation worked without a problem, as do my filters with my patch. I'm sure there are incompatibilities between the 2.4.0 library and ES 5, but maybe the things that we need won't actually hit those incompatibilities?

The filters work fine? The current format is
query:
  filtered:
    filter:
      bool:
        must: [filters from rule here]

The docs say that filtered has been deprecated. I'm not really an expert on the query DSL though.
Another breaking change: No more fields
https://github.com/Yelp/elastalert/blob/master/elastalert/elastalert.py#L243
That should be stored_fields instead.
search_type=count is removed too
https://github.com/Yelp/elastalert/blob/master/elastalert/elastalert.py#L318
Instead, you have to add size: 0
These are just what stood out when scanning the breaking changes page

@Qmando : I believe it should look like
query:
  bool:
    must: [filters from rule here]

@Qmando the filters work fine with my above patch, and the ES 2.4.0 library against a ES 5.0.0 server
The filter comes out something like:
query:
  bool:
    filter:
      bool:
        must: [filters from rule here]

It looks odd with a bool then filter then bool, it just happens we are using the bool filter.
@stumyp I'm not sure if your shorter query would work, it's not what I understood the docs to mean.

I've started a branch (https://github.com/suqld/elastalert/tree/support_es5) that we can work on. I'll try and find ways to make it crash (based on the breaking changes), then commit fixes

you're right @timwsuqld , my query would work, but a bit differently:
from documentation:
*filter*
The clause (query) must appear in matching documents.
However unlike must the score of the query will be ignored.

@Qmando Regarding the ID, it's not as easy as id=None :(
https://github.com/elastic/elasticsearch-py/issues/474#issuecomment-256903012
Looks like we need to change the call from create to index

+1

Given the number of changes, I'm wondering the best way to handle the ES5 changes. We could have a config option (or autodetect ES version) and then in all the places the query needs to be changed, have an if statement. Alternatively, we maintain 2 branches.
Suggestions?

I'm not sure how much work (and time answering people in issues) will it be to maintain everything in one place with detection/configuration option.
If it is easier to keep separate branch/tag for ES5 - I'm fine with that.

+1

+1

+1 need this for my Bachelorthesis :P

+1 would love this for our new ES5 stack.

Please don't +1 this, use the thumbs up on the issue.
The branch at https://github.com/suqld/elastalert/tree/support_es5 is currently working for me in production against ES5. I'm thinking it would be good if @stumyp could create an es5 branch that I can submit a merge request against, so we can get my changes into the elastalert repo.

@timwsuqld
Your branch seems to be working fine! Except mass amount of errors while installation process.
Btw skipped pip install -r requirements.txt in my docker container, but no problems occurred

@timwsuqld I'm external collaborator, same as you, don't have any control over this repo.
I think @Qmando can do it.

Hey guys. I took @timwsuqld's changes and added code to grab the elasticsearch version.
Please pull the support_es5 branch and test it for me!!
$ git fetch origin
$ git checkout origin/support_es5
$ pip install elasticsearch>3.0

https://github.com/Yelp/elastalert/pull/820/files

@Qmando Thanks for doing that! I'll try and test that in the next few days. I knew it should be easy to work out the version, just hadn't had time to dig that deep!

@Qmando with support_es5 I still get
ERROR:root:Error running query: TransportError(400, u'parsing_exception', u'no [query] registered for [query]')

I encountered the same problem with elmalto, while I tried to resolved it.
It was because the rule.yaml  file need to be modified ,too.
I changed it form
filter:

query:
query_string:
query: "Extends:0x60"

to
filter:

query_string:
query: "Extends:0x60"

Then it works.
It seems that the sample need to modified in  http://elastalert.readthedocs.io/en/latest/recipes/writing_filters.html#writingfilters

+1

Guys what the progress on this? We are extremely need this...

@Hronom The branch is out. I would like more feedback as I don't have an ES5 test environment right now. There are some things that need to be done, like use_terms_query doesn't work.

@Qmando So I can use branch support_es5?

It's not 100% complete, but basic functionality should work. I'm looking for feedback, so I would love it if you tried to use it and reported any issues.

Hello !
When I will install elastalert (support_es5 branch) I have this error:

error: Could not find suitable distribution for Requirement.parse('elasticsearch<3.0.0')

It's the python plugin ?

Hello !
I have an error when I launch a test:

Successfully loaded Test Rule

WARNING:elasticsearch:GET http://<IP_ADDRESS>:9200/logstash-*/snmptrap/_count?ignore_unavailable=true [status:400 request:0.003s]
Error querying Elasticsearch:
RequestError(400, u'parsing_exception', {u'status': 400, u'error': {u'line': 1, u'root_cause': [{u'reason': u'no [query] registered for [filtered]', u'type': u'parsing_exception', u'line': 1, u'col': 24}], u'type': u'parsing_exception', u'reason': u'no [query] registered for [filtered]', u'col': 24}})
Traceback (most recent call last):
File "/usr/bin/elastalert-test-rule", line 9, in
load_entry_point('elastalert==0.1.3', 'console_scripts', 'elastalert-test-rule')()
File "/usr/lib/python2.6/site-packages/elastalert/test_rule.py", line 317, in main
test_instance.run_rule_test()
File "/usr/lib/python2.6/site-packages/elastalert/test_rule.py", line 312, in run_rule_test
self.run_elastalert(rule_yaml, conf, args)
File "/usr/lib/python2.6/site-packages/elastalert/test_rule.py", line 180, in run_elastalert
load_modules(rule)
File "/usr/lib/python2.6/site-packages/elastalert/config.py", line 322, in load_modules
rule['alert'] = load_alerts(rule, alert_field=rule['alert'])
File "/usr/lib/python2.6/site-packages/elastalert/config.py", line 376, in load_alerts
alert_field = [create_alert(a, b) for a, b in alert_field]
File "/usr/lib/python2.6/site-packages/elastalert/config.py", line 367, in create_alert
return alert_class(alert_config)
File "/usr/lib/python2.6/site-packages/elastalert/alerts.py", line 311, in init
self.get_account(self.rule['smtp_auth_file'])
File "/usr/lib/python2.6/site-packages/elastalert/alerts.py", line 275, in get_account
account_conf = yaml_loader(account_file)
File "/usr/lib/python2.6/site-packages/PyStaticConfiguration-0.10.2-py2.6.egg/staticconf/loader.py", line 167, in yaml_loader
with open(filename) as fh:
TypeError: coercing to Unicode: need string or buffer, list found

I don't know if it's elastalert the problem or my configuration.

Hi,

I installed elastalert on debian:jessie with the following script:

#!/bin/bash

# misc
apt-get update
apt-get install git wget build-essential python python-pip python-dev -y
wget https://bootstrap.pypa.io/ez_setup.py -O - | python
pip install --upgrade pip
pip install --upgrade six

#elastalert
git clone https://github.com/Yelp/elastalert.git /elastalert
cd /elastalert
git checkout support_es5
python setup.py install
pip install -r requirements.txt
cp -f config.yaml.example /.backup/elastalert.yml

# purge
apt-get remove git wget build-essential python-pip python-dev -y

I am running es 5.0.2

I am executing elastalert-test-rule example.yml

example.yml:

es_host: elasticsearch
es_port: 9200
es_username: elastic
es_password: changeme
name: example
index: metricbeat-*
type: any
filter:
- range:
    system.core.cpu.idle.pct:
        from: 0
        to: 0.1
alert:
- command
command: ["echo", "\"rules.example triggered\""]

I am getting the following response:
elasticsearch:GET http://elasticsearch:9200/metricbeat-*/_search?ignore_unavailable=true&size=1 [status:400 request:0.004s]
Error running your filter:
RequestError(400, u'parsing_exception', {u'status': 400, u'error': {u'line': 1, u'root_cause': [{u'reason': u'no [query] registered for [filtered]', u'type': u'parsing_exception', u'line': 1, u'col': 68}], u'type': u'parsing_exception', u'reason': u'no [query] registered for [filtered]', u'col': 68}})

It makes sense because filtered queries do not exist anymore and have been replaced by the bool notation
GET /_search
{
  "query": {
    "bool": {
        ...
     }
}

Howerver I don't get it because it looks like you fixed it already
Any insight would be helpful.

@Qmando I've been using the support_es5 branch for the past few days without issue

Guys previously I use in rule this:
- query:
    nested:
      path: "objects"
      filter:
      - and:
        - term:
            objects.id: "12323443456757687890"
        - range:
            objects.mass:
              from: 0.012
              to: 1.0

But with this I get error:
ERROR:root:Error running query: TransportError(400, u'parsing_exception', u'no [query] registered for [query]')

What I need to change to make it work?

anyone make it work with wildcard filter?

I know this is closed but I wanted to report that the support_es5 branch, along with the changes suggested by @doublesea above to one of my alerts that had a "query" seems to be working with ES 5.1.1.

Does it normally work with ES 5.1.1 ? still waiting PR closing and merging to master. Thanks for your work guys

@i-sam, I'm not seeing any issues with ES 5.1.1 using the branch (other than the query/query_string thing described above).  That said, I only have half a dozen rules and they are pretty simple

Hi, I'm tying to use elastalert branch support_es5 against ES 5.0.2.
With the following config.yaml
# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: example_rules

# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  minutes: 1

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15

# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
es_host: localhost

# The Elasticsearch port
es_port: 9200

# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to Elasticsearch
#use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

# Email configuration
#smtp_host: "smtp.defendo.se"
#from_addr<EMAIL_ADDRESS>
# a list of email addresses to send alerts to
#email:
#-<EMAIL_ADDRESS>
test.yaml
es_host: elasticsearch
es_port: 9200
name: example
index: logs-*
type: any
filter:
- range:
    server_port:
        from: 0
        to: 80
alert:
- command
command: ["echo", "\"rules.example triggered\""]

But when running elastalert-test-rule example_rules/test.yaml we get the following error:
Successfully loaded example
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): elasticsearch
Error connecting to ElasticSearch:
UnboundLocalError("local variable 'response' referenced before assignment",)
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent. To send them, use --verbose.
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): es
Traceback (most recent call last):
  File "/usr/local/bin/elastalert-test-rule", line 11, in <module>
    load_entry_point('elastalert==0.1.4', 'console_scripts', 'elastalert-test-rule')()
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 324, in main
    test_instance.run_rule_test()
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 319, in run_rule_test
    self.run_elastalert(rule_yaml, conf, args)
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 232, in run_elastalert
    client = ElastAlerter(['--debug'])
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 116, in __init__
    self.five = self.is_five()
  File "/usr/local/lib/python2.7/dist-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 125, in is_five
    info = self.writeback_es.info()
  File "/home/defendo/.local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 71, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/defendo/.local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 222, in info
    return self.transport.perform_request('GET', '/', params=params)
  File "/home/defendo/.local/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/home/defendo/.local/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 80, in perform_request
    self.log_request_fail(method, url, response.request.path_url, body, time.time() - start, exception=e)
UnboundLocalError: local variable 'response' referenced before assignment

I don't know python but what could be the source of this issue?

That bug is from the elasticsearch python library, possibly because of a version mismatch between the cluster and the library.
What version of elasticsearch-python do you have?
pip freeze | grep elasticsearch

pip freeze | grep elasticsearch
outputs elasticsearch==5.0.1 but we have ES 5.0.2 installed could that be the reason?

If I understand the output correctly it first connects to localhost as defined in both test.yaml and config.yaml and that works as intended, then it connects to es and fails as we have no computer named es...
Where does it get the hostname es from?

That's a (very poor) default value. If config.yaml is in the same directory, it should open that up and use it. If not, you can add --config /path/to/config.yaml and it will pick it up. The whole elastalert-test-rule script needs to major work in general.
Your library version is fine, but there's still a bug in that library that is obfuscating the connection error.

For reference: elastic/elasticsearch-py#476

@Qmando , @timwsuqld : I think good way to test compatibility and see if any deprecated queries left is to run your elastalert instance on cluster with enabled deprecation logging:
curl -XPUT localhost:9200/_cluster/settings -d '{   "transient": { "logger": { "deprecation": "DEBUG" } } }'

This is what I have in my logs:
a lot of this:
[2016-12-28 16:01:55,259][DEBUG][deprecation.common       ] Deprecated field [ignore_unmapped] used, replaced by [unmapped_type]

and this:
[2016-12-28 16:01:56,658][DEBUG][deprecation.index.query  ] The [query] filter is deprecated, you can now use queries as filters directly.
[2016-12-28 16:01:56,658][DEBUG][deprecation.index.query  ] The [and] query is deprecated, please use a [bool] query instead with [must] clauses.
[2016-12-28 16:01:56,658][DEBUG][deprecation.index.query  ] The [filtered] query is deprecated, please use a [bool] query instead with a [must] clause for the query part and a [filter] clause for the filter part.

@Qmando Thanks adding --config /path/to/config.yaml resolved the issue.

@tfgm-bud  What elasticsearch python module version are you using?  Also, can you post some of those simple queries.  Thank You!

@bHoskins07 Not sure what you are asking with "What elasticsearch python module version are you using?".  I'm using the github version and changing to the support_es5 branch:
git checkout origin/support_es5
And a simple query would be:
filter:
- term:
    env: "live"
- query_string:
    query: 'msgtype:activity AND ("transaction" adjusted unchanged cancelled)'

Not sure how much that really helps you though....

@tfgm-bud To find the version of the elasticsearch module, type $ pip freeze | grep elasticsearch

@Qmando oh, I mis-read @bHoskins07 question and was thinking he was asking about elastalert.
elasticsearch==2.4.0

Im new to this but why does this work:
filter:
- query_string:
    query: "received_from: Prod AND (level: FATAL OR level: ERROR)"

But this doesn't:
filter:
- and:
    - term:
        received_from: "Prod"
    - or:
        - term:
            level: "FATAL"
        - term:
            level: "ERROR"

Aren't they the same thing?
The latter gives the following error {'message': "Error running query: TransportError(400, u'parsing_exception', u'[and] query malformed, no start_object after query name')"...
I'm guessing this is the same issue as @Hronom has.

Thank You @tfgm-bud.  I was testing 5.1.1 using  elastalert-test-rule which was not working.  When I switched to $ python -m elastalert.elastalert --rule it worked.  Also, I am using python elasticsearch module 5.0.1

@bHoskins07 - I see what you mean.  I upgraded to elasticsearch-5.0.1 with sudo pip install --upgrade elasticsearch and elastalert-test-rule fails:
$ elastalert-test-rule --config /etc/elastalert/config.yaml /etc/elastalert/rules/news.yaml
Traceback (most recent call last):
  File "/usr/local/bin/elastalert-test-rule", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3015, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2999, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3028, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 657, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 670, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 849, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'elasticsearch<3.0.0' distribution was not found and is required by elastalert

I made the following change to try to fix it and it works for me:
user@elk:~/elastalert[support_es5]$ git diff
diff --git a/setup.py b/setup.py
index 1f433c1..ebf817a 100644
--- a/setup.py
+++ b/setup.py
@@ -23,7 +23,7 @@ setup(
     package_data={'elastalert': ['schema.yaml']},
     install_requires=[
         'argparse',
-        'elasticsearch<3.0.0',  # Elastalert is not yet compatible with ES5
+        'elasticsearch',
         'jira==0.32',  # jira.exceptions is missing from later versions
         'jsonschema',
         'mock',

@timwsuqld can you incorporate this into your branch?  -- Bud

@Qmando , I had the same issue while working with master branch. After checking out support_es5 branch and switching to that branch, it works perfectly fine.  Can you please merge this branch to master  branch and let me know after merging?

Wahoo! I'm watching and waiting for ES5!

I added an issue with the branch as a comment in the PR. Is that OK? Should I mention the problem here or should I create a separate issue for that branch?

Hi,
Yes, thanks for adding it in the PR. No need to create a separate issue for that branch.
Thanks and regards,
Deepthi D
From: Dominik Sandjaja<EMAIL_ADDRESS>Sent: Monday, January 9, 2017 8:53 PM
To: Yelp/elastalert<EMAIL_ADDRESS>Cc: Deepthi D<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [Yelp/elastalert] Integration with elasticsearch v5 (#790)
I added an issue with the branch as a comment in the PR. Is that OK? Should I mention the problem here or should I create a separate issue for that branch?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/Yelp/elastalert/issues/790#issuecomment-271311477, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AWWRWKEbKy_zcbgsbmW228UAHZqtCuVMks5rQlC4gaJpZM4KnOvN.

then, I would try elastalert on ES5?

+1

@rptete Go right ahead.

+1. is there an eta for merging the support_es5 branch into mainstream?

Please notify when you merge this branch to the master one. Thanks

searchguard5 with compatibility to ES-5.1.2 and SGadmin10:
wget http://repo1.maven.org/maven2/com/floragunn/search-guard-5/5.1.2-10/search-guard-5-5.1.2-10.zip
/usr/share/elasticsearch/bin/elasticsearch-plugin install file:///root/search-guard-5-5.1.2-10.zip
/usr/share/elasticsearch/bin/elasticsearch-plugin list
searchguard5-kibanaPlugin with compatibility to KI-5.1.2
wget https://github.com/floragunncom/search-guard-kibana-plugin/releases/download/v5.1.2-alpha/searchguard-kibana-alpha-5.1.2.zip
/usr/share/kibana/bin/kibana-plugin install file:///root/searchguard-kibana-alpha-5.1.2.zip
/usr/share/kibana/bin/kibana-plugin list
4u players :)
Thanks to Marc Escamilla

+1

This has been merged with master and released.

Im running elasticsearch 5.2.2
wen I run elastalert-create-index i have this error:
root@logstash:/data/elastalert# elastalert-create-index
Traceback (most recent call last):
File "/usr/local/bin/elastalert-create-index", line 6, in
from pkg_resources import load_entry_point
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 3037, in
@_call_aside
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 3021, in _call_aside
f(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 3050, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 655, in _build_master
ws.require(requires)
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 969, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 855, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'jira>=1.0.10' distribution was not found and is required by elastalert
root@logstash:/data/elastalert#

after
pip install -r requirements.txt
pip install functools32
its working, thanks

Hi everyone
Im running elasticsearch 5.3.0 with windows
wen I run elastalert-test-rule  i have this error:
C:\Python27\Scripts>elastalert-test-rule example_rules/4.yaml Traceback (most recent call last):   File "C:\Python27\Scripts\elastalert-test-rule-script.py", line 11, in <module>     load_entry_point('elastalert==0.1.6', 'console_scripts', 'elastalert-test-rule')()   File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 565, in load_entry_point     return get_distribution(dist).load_entry_point(group, name)   File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2631, in load_entry_point     return ep.load()   File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2291, in load     return self.resolve()   File "C:\Python27\lib\site-packages\pkg_resources\__init__.py", line 2297, in resolve     module = __import__(self.module_name, fromlist=['__name__'], level=0)   File "C:\Python27\lib\site-packages\elastalert-0.1.6-py2.7.egg\elastalert\test_rule.py", line 20, in <module>     import elastalert.config   File "C:\Python27\lib\site-packages\elastalert-0.1.6-py2.7.egg\elastalert\config.py", line 8, in <module>     import alerts   File "C:\Python27\lib\site-packages\elastalert-0.1.6-py2.7.egg\elastalert\alerts.py", line 26, in <module>     from twilio import TwilioRestException ImportError: cannot import name TwilioRestException


Timed out waiting for https://rerun.io/viewer/commit/bc8bcd0 to load.

When executing rr.show_notebook(), we always get a cell showing:
Timed out waiting for https://rerun.io/viewer/commit/bc8bcd0 to load.

For locally run notebooks, the workaround:
rr.start_web_viewer_server()

will make things for.
For colab notebooks we need a proper fix (https://github.com/rerun-io/rerun/pull/5877)

That's already been fixed and merged into the 0.15.1 branch.


Fix libdevmapper deferred removal detection

When linking, position of -l flags is important since
they muse come after any object files which uses symbols
from a specified library, that is due to --as-needed binutils
ld flag enabled by default

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "fix-deferred-removal-detection" git@github.com:redbaron/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Job: Docker-PRs-Win2Lin FAILED:

---
-forward
 hack/make.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
[docker] $ sh -xe C:\Users\Jenkins\AppData\Local\Temp\hudson5286950867294109853.sh
+ set +x
INFO: Started at Thu Feb 18 00:00:08 CUT 2016. Script version 4-Jan-2016 15:19 PST
Build step 'Execute shell' marked build as failure
Performing Post build task...
Could not match :Build timed out  : False
Logical operation result is FALSE
Skipping script  : cd c:\scripts
sh timeout-post-build.sh

END OF POST BUILD TASK 	: 0
[PostBuildScript] - Execution post build scripts.
[docker] $ sh -xe C:\Users\Jenkins\AppData\Local\Temp\hudson7571911143181297126.sh
+ set +x

---

/cc @tianon, @jfrazelle

Nice, seems sane!
LGTM

Just to clarify, current build process always reports that deferred removal/deletion isn't available in libdevmapper and, therefore, disables these features in devicemapper storage driver

LGTM

can it be cherry-picked into 1.10.2 branch too pls?

Let me add it to 1.10.3 for consideration (and if 1.10.3 is released at all)

Just to clarify, current build process always reports that deferred removal/deletion isn't available in libdevmapper and, therefore, disables these features in devicemapper storage driver

@thaJeztah @crwd-mivanov how does this happens? we're not experiencing this - we build and enable the feature just fine as it was before this patch

@runcom I can think of 2 cases why it can work for you:

you are building it in the way, that hack/make.sh isn't used or DOCKER_BUILDTAGS are passed externally with libdm_no_deferred_remove value in it
linker you are using defaults to '--no-as-needed' and therefore linker is happy to overlink making pre-patch test succeed, as order isn't important in that case

linker you are using defaults to '--no-as-needed' and therefore linker is happy to overlink making pre-patch test succeed, as order isn't important in that case

I think we build with 2) then, how can I verify this?

Another option is that you set/override LD_FLAGS somewhere in your build process.
I don't know clean way to make linker to report whether it is using --as-needed  or not, but you can check it indirectly by doing something similar to what is happening make.sh:
gcc -o /tmp/test -xc -  -lssl <<< "int main() {return 0;}" && readelf -d /tmp/test|grep ssl

if you see libssl in the output, then linker is overlinking (== using --no-as-needed)

 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.10]

this is the output on my f23 box

So GNU ld in f23 is still in ancient times then :)


refactor: reorganize beatree update modules

extract update stages to their own modules.
rename branch and leaf updater modules to branch_updater and leaf_updater
nit: stop exporting branch_updater and leaf_updater publicly

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#391  👈
#382
#381
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rphmeier and the rest of your teammates on  Graphite

Merge activity

Sep 6, 7:06 PM EDT: @rphmeier started a stack merge that includes this pull request via Graphite.


esp_restart() crash on rtc_clk_cpu_freq_to_xtal since 6b0a5af73 (IDFGH-4466)

Environment

Development Kit:      ESP32-Wroom-32D rev1 on custom board, with ethernet
IDF version (run git describe --tags to find it):
problem present since commit 6b0a5af73, still present on master
Build System:       make
Operating System:    Linux
Using an IDE?: atom

Problem Description
On esp_restart(), it goes into bootloop.
Expected Behavior
Reboot, not to bootloop
Actual Behavior
bootloop
`GPIO[39] intr, val: 0
W (33166) main-task: RESET NETWORK CONFIG and reboot
I (33766) wifi:flush txq
I (33766) wifi:stop sw txq
I (33766) wifi:lmac stop hw txq
W (33766) main-task: esp_restart called, reboot
Re-enable cpu cache.
Guru Meditation Error: Core  1 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x4010fa8c: 284aacb9 a502ad01 0acd30db
0x4010fa8c: rtc_clk_cpu_freq_to_xtal at /home/lorenzo/esp/esp-idf/components/esp_hw_support/port/esp32/rtc_clk.c:474
Core  1 register dump:
PC      : 0x4010fa92  PS      : 0x00060330  A0      : 0x8008145c  A1      : 0x3ffcdcc0
0x4010fa92: rtc_clk_cpu_freq_set_xtal at /home/lorenzo/esp/esp-idf/components/esp_hw_support/port/esp32/rtc_clk.c:545
A2      : 0x00000000  A3      : 0x3ff000c4  A4      : 0x00000001  A5      : 0x00000001
A6      : 0x10624dd3  A7      : 0x00000001  A8      : 0x80084c33  A9      : 0x3ffcdca0
A10     : 0x3801ff80  A11     : 0x3801ff80  A12     : 0x000249f0  A13     : 0x3801fb80
A14     : 0x3f400ab8  A15     : 0x3ffcd5e0  SAR     : 0x0000001e  EXCCAUSE: 0x00000000
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffd
Backtrace:0x4010fa8f:0x3ffcdcc0 0x40081459:0x3ffcdd00 0x400d3b4d:0x3ffcdd20 0x4008bfdd:0x3ffcdd50
0x4010fa8f: rtc_clk_cpu_freq_to_xtal at ??:?
0x40081459: esp_restart at /home/lorenzo/esp/esp-idf/components/esp_system/system_api.c:83
0x400d3b4d: reset_button_task at /home/lorenzo/Desktop/lavoro/Progetti/ElektraConnect/doModem/main/domodem_main.c:71
0x4008bfdd: vPortTaskWrapper at /home/lorenzo/esp/esp-idf/components/freertos/port/xtensa/port.c:168
ELF file SHA256: 7998f07ef4d4c8fe
Rebooting...
Re-enable cpu cache.
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core  1 register dump:
PC      : 0x40116f27  PS      : 0x00060a33  A0      : 0xf5a5fe6a  A1      : 0x3ffcdae0
0x40116f27: hostap_input at ??:?
`
end so on:
`ELF file SHA256: 7998f07ef4d4c8fe
Rebooting...
Re-enable cpu cache.
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core  1 register dump:
PC      : 0x40116f27  PS      : 0x00060633  A0      : 0xf5a5fe6a  A1      : 0x3ffc4570
0x40116f27: hostap_input at ??:?

ELF file SHA256: 7998f07ef4d4c8fe
Rebooting...
Re-enable cpu cache.
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core  1 register dump:
PC      : 0x40116f27  PS      : 0x00060f33  A0      : 0xf5a5fe6a  A1      : 0x3ffc4390
0x40116f27: hostap_input at ??:?
A2      : 0x00000001  A3      : 0x00000001  A4      : 0x00000009  A5      : 0x00000009
A6      : 0x20100000  A7      : 0x60000000  A8      : 0x8010faa0  A9      : 0x3ffc4360
A10     : 0x3f44ccf4  A11     : 0x00000001  A12     : 0x00000001  A13     : 0x3ffc4370
A14     : 0x00000018  A15     : 0x3ffb0040  SAR     : 0x00000000  EXCCAUSE: 0x0000001c
EXCVADDR: 0x8010faa0  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000
Backtrace:0x40116f24:0x3ffc4390 0x75a5fe67:0x3ffc43b0 |<-CORRUPTED
0x40116f24: hostap_input at ??:?
`
Steps to reproduce
I don't have another shield with ethernet (I don't know why but I think ethrnet is kind of related with this problem, maybe for the clock on gpio0) to try if it happents also there.
I'm pretty sure that the problem is present since this commit: 6b0a5af73
I don't want to keep the idf not updated after the latest working commit, so I would like to fix this. How can I help you? I can provide all the informations you need.
I want to point out that on older esp-idf commits, I have no problem and it reboot. Do you think this may also be related with esp chip revision?
Thank you

Thanks for reporting and sorry for inconvenience, we will look into.

Hi @lorenzomerici,
Sorry for the late reply. Can you check if adding COMPONENT_ADD_LDFRAGMENTS := linker.lf to $IDF_PATH/components/esp_hw_support/component.mk fixes the issue?

Hi @lorenzomerici,
Sorry for the late reply. Can you check if adding COMPONENT_ADD_LDFRAGMENTS := linker.lf to $IDF_PATH/components/esp_hw_support/component.mk fixes the issue?

Yes you were right! This fix the issue. I should keep it like this or you will update the idf soon?

Yes you were right! This fix the issue. I should keep it like this or you will update the idf soon?

May I know what kind of condition will triger this issue?
Or which configuration causes this issue?
Just curious why my device using master tree does not hit this crash.

May I know what kind of condition will triger this issue?
Or which configuration causes this issue?
Just curious why my device using master tree does not hit this crash.

Maybe it's related with the hardware rev? I'm current on REV1. Let's wait if @renzbagaporo can give us more info

Maybe it's related with the hardware rev? I'm current on REV1. Let's wait if @renzbagaporo can give us more info

This was a regression related to code placement. The functions in question were placed in flash instead of RAM.

This was a regression related to code placement. The functions in question were placed in flash instead of RAM.


ExpressJS failing to start

Hi I am trying to swap out the ExpressJS default hbs router for express-handlebars, yet when I try to do it I get the following error when trying to access a page. What have I done wrong?
Error
   at Object.<anonymous> (/Users/custa/github/Portfolio/portfolioapp/node_modules/hbs/node_modules/handlebars/dist/cjs/handlebars/exception.js:29:23)
   at Module._compile (module.js:434:26)
   at Object.Module._extensions..js (module.js:452:10)
   at Module.load (module.js:355:32)
   at Function.Module._load (module.js:310:12)
   at Module.require (module.js:365:17)
   at require (module.js:384:17)
   at Object.<anonymous> (/Users/custa/github/Portfolio/portfolioapp/node_modules/hbs/node_modules/handlebars/dist/cjs/handlebars/base.js:3:17)
   at Module._compile (module.js:434:26)
   at Object.Module._extensions..js (module.js:452:10)
   at Module.load (module.js:355:32)
   at Function.Module._load (module.js:310:12)
   at Module.require (module.js:365:17)
   at require (module.js:384:17)
   at Object.<anonymous> (/Users/custa/github/Portfolio/portfolioapp/node_modules/hbs/node_modules/handlebars/dist/cjs/handlebars.runtime.js:3:12)
   at Module._compile (module.js:434:26)

I have narrowed this down and it's if I include ad use a route file i.e.:
var express = require('express');
var path = require('path');
var exphbs  = require('express-handlebars');

var app = express();

// Routes
var test = require('./routes/test');

var hbs = exphbs.create({
    defaultLayout: 'main',
    extname: ".hbs",
    layoutsDir: path.join(__dirname, "views/layouts/"),
    partialsDir: path.join(__dirname, 'views/partials')
});
// Register `hbs.engine` with the Express app.
app.engine('handlebars', hbs.engine);
app.set('view engine', 'hbs');
app.set('views', path.join(__dirname, 'views'));
app.use(express.static(path.join(__dirname, 'public')));
app.use('/bower_components', express.static(__dirname + '/../bower_components'));

app.use("/", test);

app.get('/', function (req, res) {
  res.send('Hello World!');
});

app.listen(3001, function () {
    console.log('express-handlebars example server listening on: 3001');
});

module.exports = app;

Even with a simple get in that route file it throws the original error. The route file looks like the following:
var express = require('express');
var router = express.Router();

router.get('/', function(req, res) {
    res.render('about', {
        title: 'About'
    });
});

module.exports = router;

Throws the following errors:
Error
   at Object.<anonymous> (/Users/custa/github/Portfolio/portfolioapp/node_modules/hbs/node_modules/handlebars/dist/cjs/handlebars/exception.js:29:23)
   at Module._compile (module.js:434:26)
   at Object.Module._extensions..js (module.js:452:10)
   at Module.load (module.js:355:32)
   at Function.Module._load (module.js:310:12)
   at Module.require (module.js:365:17)
   at require (module.js:384:17)
   at Object.<anonymous> (/Users/custa/github/Portfolio/portfolioapp/node_modules/hbs/node_modules/handlebars/dist/cjs/handlebars/base.js:3:17)
   at Module._compile (module.js:434:26)
   at Object.Module._extensions..js (module.js:452:10)

Is it possible that I am running too new a version of Node?

My first thought: perhaps the error is caused by importing the routes before defining the rendering engine? Try moving var router = require('./router')(app); below app.engine('handlebars', hbs.engine); and the lines that follow.

{{#contentBlock "head"}}
{{/contentBlock}}

{{>hero/home }}
{{>section/about}}
{{>section/skills}}
{{>section/linkedin}}

{{#contentBlock 'scripts'}}
{{/contentBlock}}

@custa1200 Thanks! This is very much a "turn it off / back on again" approach to your problem, but have you tried removing all the Handlebars logic from that view, just to see if it still errors even without having to parse any template markup? I've had my code break due to a minor syntax error in my Handlebars markup, so sometimes starting from scratch and slowly re-adding code has been helpful for me.

Looks like a had a mismatch which caused the issues with
app.engine('handlebars', hbs.engine);
app.set('view engine', 'hbs');

worked when I made it
app.engine('hbs', hbs.engine);
app.set('view engine', 'hbs');