Getting 'monitor_window_timeout' and 'X-Frame-Options set to deny' error when login with facebook account

Core Library
MSAL.js v2 (@azure/msal-browser)
Core Library Version
2.22.0
Wrapper Library
MSAL React (@azure/msal-react)
Wrapper Library Version
1.3.0
Description
We're getting an error sometimes when login with facebook. It's hard to say how this error is triggered, it looks like when we change between our applications or environments the error occurs but I don't know the real trigger. Apparently this error is prompted only with facebook accounts and if I clear all the application cache storage and refresh the page then I am able to login correctly with facebook account.
Error Message
Refused to display 'https://www.facebook.com/' in a frame because it set 'X-Frame-Options' to 'deny'.

Uncaught (in promise) Error: BrowserAuthError: monitor_window_timeout: Token acquisition in iframe failed due to timeout. For more visit: aka.ms/msaljs/browser-errors.

Msal Logs

MSAL Configuration
{
  auth: {
    clientId: process.env.REACT_APP_AZURE_B2C_CLIENT_ID,
    authority: b2cPolicies.authorities.signUpSignIn.authority,
    knownAuthorities: [
      b2cPolicies.authorityDomain,
      b2cPolicies.authorities.signUpSignIn.authority,
      b2cPolicies.authorities.changePassword.authority,
      b2cPolicies.authorities.editProfile.authority,
    ],
    redirectUri: process.env.REACT_APP_AZURE_B2C_REDIRECT_URI,
    postLogoutRedirectUri: process.env.REACT_APP_AZURE_B2C_POST_LOGOUT_URI,
  },
  cache: {
    cacheLocation: 'localStorage',
  },
  system: {
    loggerOptions: {
      loggerCallback: (level, message, containsPii) => {
        if (containsPii) {
          return;
        }
        switch (level) {
          case LogLevel.Error:
            console.error(message);
            return;
          case LogLevel.Info:
            console.info(message);
            return;
          case LogLevel.Verbose:
            console.debug(message);
            return;
          case LogLevel.Warning:
            console.warn(message);
        }
      },
    },
  },
}

Relevant Code Snippets
Here is how we get the token silently

instance
          .acquireTokenSilent({
            authority: b2cPolicies.authorities.signUpSignIn.authority,
            scopes: ['openid', process.env.REACT_APP_AZURE_B2C_SCOPES],
            account,
            forceRefresh: true,
          })
          .then(response => {
            if (response) {
              setUser({
                ...account.idTokenClaims,
                token: response.accessToken,
              });
            }
          })
          .catch(err => {
            if (err instanceof InteractionRequiredAuthError) {
              const request = {
                authority: b2cPolicies.authorities.signUpSignIn.authority,
                scopes: loginRequest.scopes,
                prompt: 'login',
              };
              return instance.acquireTokenRedirect(request);
            }
            throw Error(err);
          });

Reproduction Steps

Go to the application url and get redirected to SSO login page
Login with facebook account
SSO redirects back to the application with the error prompted in console

Expected Behavior

Go to the application url and get redirected to SSO login page
Login with facebook account
SSO redirects back to the application and retrieve the token correctly

Identity Provider
Azure B2C Custom Policy
Browsers Affected (Select all that apply)
Chrome, Firefox
Regression
No response
Source
External (Customer)

Hi, could you email your network trace to the email included on my github profile? We believe this is because the b2c policy is not forwarding the login hint.

@mpminayo I couldn't find an e-mail in your profile

Hi, could you email your network trace to the email included on my github profile? We believe this is because the b2c policy is not forwarding the login hint.

@mpminayo I couldn't find an e-mail in your profile

Sorry, check now.

I've fixed the email issue, but you can send to @pkanher617.

I've fixed the email issue, but you can send to @pkanher617.

@mpminayo  I sent to your e-mail our network and console logs for the error 👍

Hi @ranzolinrafa, I can see in the logs that there is the following Bad Request error:
XhrClient.js:120          POST https://loginb2cnonprod.b2clogin.com/loginb2cnonprod.onmicrosoft.com/b2c_1a_dtc_custom_lounge_signuporsignin/oauth2/v2.0/token 400 (Bad Request)

However I can't find any info about this in the sent logs, are there any browser console logs that tell us why this request failed?
The error initially reported is most likely happening on the B2C service, but that is happening after the Bad Request error, so that may be the root cause.


[#3362] feat(flink-connector): Add the code skeleton for flink-connector

What changes were proposed in this pull request?

support GravitinoCatalogStore to register the catalog. In the MR, we will support to create the hive catalog.

Why are the changes needed?

Fix #3362

Does this PR introduce any user-facing change?

support flink in gravitino

How was this patch tested?

add UTs

@coolderli can we please move forward the flink support?

@coolderli can we please move forward the flink support?

@jerryshao Of course. I will finish the first one today.

@FANNG1 Could you help review this MR again? Thanks.

@FANNG1 I think I have already rebased the main branch. But the GitHub CI still failed. Can you help me with it? Thanks.

seems CI failure are cause by #3390 which change the API about metalake and catalog operations. You could refer the latest the implement of Spark GravitinoCatalogManager for the usage.

@FANNG1 I have fixed the pipeline. Please help review again. Thanks.

@FANNG1 Address all comments. Please review it again. Thanks a lot.

LGTM, except few comments, @jerryshao do you have time to review the PR?

@FANNG1 You can go ahead if you feel OK. Thanks @coolderli for the first step of Flink support.

@coolderli , thanks for proposing the base PR for Flink, a big step!


SDCCC-1211 Fix BICEP preconditions

SDCCC-1211 Fix BICEP preconditions
Checklist
The following aspects have been respected by the author of this pull request, confirmed by both pull request assignee and reviewer:

Adherence to coding conventions

[X] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

Adherence to javadoc conventions

[X] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

Changelog update (necessity checked and entry added or not added respectively)

[X] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

README update (necessity checked and entry added or not added respectively)

[X] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

config update (necessity checked and entry added or not added respectively)

[X] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

SDCcc executable ran against a test device (if necessary)

[ ] Pull Request Assignee
[ ] Reviewer Max
[ ] Reviewer Malte

please edit the checklist of the initial comment to make it adhere to the fact that there are two reviewers now
also there is a whitespace inside one of the checkboxes and thus it is not displayed correctly

Done

Only one of the currently present 3 commits shows as verified.

Reconsidered some changes and accidentally used the main branch. Will create new PR's for each testcase separately

Reconsidered some changes and accidentally used the main branch. Will create new PR's for each testcase separately


Remove the useless logstream.Start

Proposed Changes

This PR removed the useless logstream.

/lgtm
/approve


Support multiple sideCommand items.

Sometimes your installation needs several applications to run properly.

It would be cool to do it as an array of processes, so you could just have as many as you want.


background-color not working when using certain CSS properties while cascading styles

Problem
While cascading styles in different html tags, when I use the CSS properties: font-weight, text-decoration or color, the background-color property stops working. However everything else seems to work with cascading styles.
Example

Normal

Bold

Italic

Bold+Italic

Line through

Underline

Line through + Underline

Blue text

Result

As you can see, the background-color property is set to #ffff00, which should result in a yellow background color for all elements with the class "highlightClass". However, when I apply any of the mentioned CSS properties to these elements, the background-color property stops working.

Interesting. It seems the styles are not being inherited. I'm trying to determine if this is an issue with react-pdf

It looks like the react-pdf-html rendering is resulting in this for the last paragraph in your example:
<View>
  <Text>
    <Text
      style={[
        {
          margin: '0',
        },
        {
          backgroundColor: '#ffff00',
        },
      ]}
    >
      <Text
        style={[
          {
            color: 'blue',
          },
          {
            margin: '0',
          },
        ]}
      >
        Blue text
      </Text>
    </Text>
  </Text>
</View>

The extra Text wrapper causes a loss of the background color in react-pdf. I'm not yet sure why.

This is a reported bug with react-pdf:
https://github.com/diegomura/react-pdf/issues/2316
...BUT I just published an update with a fix for your exact issue. Enjoy!


Error in maven build of code

mvn clean install - results in ->
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 1.421 s <<< FAILURE! - in oracle.kubernetes.operator.HealthCheckHelperTest [ERROR] testAccountNoPrivs(oracle.kubernetes.operator.HealthCheckHelperTest)  Time elapsed: 1.42 s  <<< FAILURE! java.lang.AssertionError: Log output did not contain Access Denied error 	at oracle.kubernetes.operator.HealthCheckHelperTest.testAccountNoPrivs(HealthCheckHelperTest.java:108)

Bryan, what's your k8s server version?

This appears to have been fixed. Can you please refresh to the latest version and try again?


Make elastix install more user-friendly for mac

See thread https://forum.image.sc/t/registration-of-whole-slide-images-at-cellular-resolution-with-fiji-and-qupath/51379

Already fixed a while ago and in different repos


Update EdgeView to 0.9985

Thank you for the contribution. It was merged directly as cc18f081ed7c5d89059dc1cd13c322855c8635e7 to keep commit history cleaner. Your contribution is still credited to you.


Lb/remove mentors from provider primary nav

Context
We are removing "Mentors" from the primary navigation for provider users because we do not have a use case for when provider users will view mentors.
Changes proposed in this pull request

Separate schools / provider primary navigation into different components.

Guidance to review
Sign on as a user who belongs to a provider.
click around and make sure you never see "Mentors" in the primary nav
Link to Trello card
https://trello.com/c/gkGzwwrD
Things to check

[ ] If the code removes any existing feature flags, a data migration has also been added to delete the entry from the database
[ ] This code does not rely on migrations in the same Pull Request
[ ] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[ ] If this code adds a column to the DB, decide whether it needs to be in analytics yml file or analytics blocklist
[ ] API release notes have been updated if necessary
[ ] If it adds a significant user-facing change, is it documented in the CHANGELOG?
[ ] Required environment variables have been updated or added to the Azure KeyVault

Screenshots
Before:

After:

 Remove Mentors from Provider Navigation


allow to extends Gesdinet\JWTRefreshTokenBundle\Entity\RefreshToken …

... keeping ORM base annotations working.
When overriding Gesdinet\JWTRefreshTokenBundle\Entity\RefreshToken with another class (to customize the table name actually), extending Gesdinet\JWTRefreshTokenBundle\Entity\RefreshToken because all class properties are private. And I don't want to copy&paste the whole code

:+1:

:+1:


Custom CA For Rancher Cluster

I have noticed that the Rancher UI has an option for a Custom CA, not sure if this is for the cattle system (aka cluster nodes) or that field strictly for the ingress certs?
Im asking this because our compliance framework requires that there are no self-signed certs, thus the internal signing CA that is shipped with Rancher is failing compliance.

Monthly checkin.

Monthly checkin.

Monthly checkin?

Monthly Checkin?

Monthly checking

Monthly checkin?


feat: atomic ntoken

Signed-off-by: GopherJ<EMAIL_ADDRESS>Security Checklist

[ ] 1. Re-Entrancy
[ ] 2. Arithmetic Over/Under Flows
[ ] 3. Unexpected Ether
[ ] 4. Delegatecall
[ ] 5. Default Visibilities
[ ] 6. Entropy Illusion
[ ] 7. External Contract Referencing
[ ] 8. Short Address/Parameter Attack (off chain)
[ ] 9. Unchecked CALL Return Values
[ ] 10. Race Conditions / Front Running
[ ] 11. Denial Of Service (DOS)
[ ] 12. Block Timestamp Manipulation
[ ] 13. Constructors with Care
[ ] 14. Uninitialized Storage Pointers
[ ] 15. Floating Points and Precision
[ ] 16. Tx.Origin Authentication
[ ] 17. Address.isContract Re-Entrancy via Constructor

⚠ NOTES ⚠
Make sure to think about each of these exploits in this PR.

@GopherJ I did these operation, but compile error happened.
git checkout main
git pull
git merge --squash feat/atomic-ntoken
make build


Support for Zeebe 1.0.0

Hi Pierre-Louis, Zeebe 1.0.0 is coming out - it's roughly expected around April. It looks like it will contain a breaking change to the client API.
The client API change hasn't been merged yet, but I've documented it in this post.

I can confirm, that with current version of zeebe-client (0.16.0) the beez gem not compatible.

I've worked on some changes on a forked repo. However it's more than just supporting the breaking changes (also added OAuth connection). @gottfrois would you be interested in a PR to this gem or are you not really maintaining this gem anymore? It would definitely need a review since gRPC and Zeebe was all new to me until a couple weeks ago.

Hey folks,
I'm not actively maintaining the gem but I'll accept PRs for sure :)
The client part was supported by zeebe-client gem actually so we should be looking there to update I think.

Took some time to see what were the changes. I think we can easily support the breaking change in the 1.0.0 API and handle the OAuth authentication @lienvdsteen mentioned in a totally different PR.

I've quickly merged the following PR in master. I'd like to do a full feature test before bumping the major version number.
https://github.com/gottfrois/beez/pull/15
Might as well spend some time adding specs... finally 😂


hideable directory path

Issue Type: Feature Request
Hello! I have this issue where my directory path is horribly long and every time I run the code that I've been running, it first prints out two different directory paths, then my code and then a last directory path in the end of it all. It would be nice if it were possible to have options or a separate terminal that only shows what the program is supposed to be printing or running at the moment and the error messages ofcourse.
VS Code version: Code 1.53.1 (5d424b828ada08e1eb9f95d6cb41120234ef57c7, 2021-02-08T23:29:42.785Z)
OS version: Windows_NT x64 10.0.18363

You can do this by modifying your shell settings, for example on bash you could edit the $PS1 variable.


cy.task is not a function

I've followed the instructions from the readme, but I'm getting the following.
cypress_runner.js:139349 TypeError: cy.task is not a function
    at Context.matchImageSnapshot (http://localhost:8081/__cypress/tests?p=tests/e2e/support/index.js-677:26:8)
From previous event:
    at runCommand (http://localhost:8081/__cypress/runner/cypress_runner.js:63043:14)
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:63125:14)
From previous event:
    at http://localhost:8081/__cypress/runner/cypress_runner.js:63144:37
From previous event:
    at run (http://localhost:8081/__cypress/runner/cypress_runner.js:63142:15)
    at Object.cy.(anonymous function) [as matchImageSnapshot] (http://localhost:8081/__cypress/runner/cypress_runner.js:63362:11)
    at Context.runnable.fn (http://localhost:8081/__cypress/runner/cypress_runner.js:63493:20)
    at callFn (http://localhost:8081/__cypress/runner/cypress_runner.js:33042:21)
    at Test.Runnable.run (http://localhost:8081/__cypress/runner/cypress_runner.js:33035:7)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:66382:28
From previous event:
    at Object.onRunnableRun (http://localhost:8081/__cypress/runner/cypress_runner.js:66381:20)
    at $Cypress.action (http://localhost:8081/__cypress/runner/cypress_runner.js:61845:51)
    at Test.Runnable.run (http://localhost:8081/__cypress/runner/cypress_runner.js:65521:20)
    at Runner.runTest (http://localhost:8081/__cypress/runner/cypress_runner.js:33505:10)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33611:12
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:33425:14)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33435:7
    at next (http://localhost:8081/__cypress/runner/cypress_runner.js:33367:14)
    at http://localhost:8081/__cypress/runner/cypress_runner.js:33403:5
    at timeslice (http://localhost:8081/__cypress/runner/cypress_runner.js:28644:27)

I'm using Vue's CLI and its plugin for cypress, if that's of any help.
Here are my complete files which I think are relevant. The project is private so I cannot share it, but I can try reproducing later if needed -- thought it might be obvious from the stack trace to the maintainers, so wanted to avoid setting everything up.
plugins/index.js
const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin')
module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on)
  return Object.assign({}, config, {
    fixturesFolder: 'tests/e2e/fixtures',
    integrationFolder: 'tests/e2e/specs',
    screenshotsFolder: 'tests/e2e/screenshots',
    videosFolder: 'tests/e2e/videos',
    supportFile: 'tests/e2e/support/index.js'
  })
}

(The return and Object.assign were there form Vue's scaffolding).
support/commands.js
import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command'
addMatchImageSnapshotCommand()

(This file was previously empty from Vue's scaffolding)
specs/public-pages.js
describe(`Public pages`, () => {
    before(() => {
        cy.visit('/') })

    describe(`Login page`, () => {
        it(`is the first page user sees when not logged in`, () => {
            cy.url().should('contain', '/login') })
        it(`looks as per design`, () => {
            cy.matchImageSnapshot() })
        it(`should have a link to Forgot password page`, () => {
            cy.get('a[href="/forgot-password"]')
                .should('exist')
                .contains(`Forgot Password?`)
                .click() }) })

    describe(`Forgot Password page`, () => {
        it(`opens after clicking the link from Login`, () => {
            cy.url().should('contain', '/forgot-password') })
        it(`looks as per design`, () => {
            cy.matchImageSnapshot() }) }) })

And here are the results:

Which version of Cypress are you using?  cy.task was added in 3.0.0 so that could be the issue.

I'll have to check it out in details. The Cypres version is, unfortunately, hidden behind a Vue CLI plugin.

Yup, after upgrading the Cypress plugin, it installed 3.1.0 of Cypress and it's all working now. Thank you for the quick response!


RFC 6750

I used your last commit to passport-google-token since I don't want to duplicate your work.
For now you can apply and push it again, or wait till your would be merged and I'll do it over your.


Override existing validator

Hi @joeybloggs ,
I really liked the validator. great work!
One thing which i don't know whether exist or not that can we override existing validation for any type.
for example: for "Required" tag i want that if user doesn't enter anything then it should fail + when user enters white character then also it should fail for my requirements. can i override the required tag validation so that it will do validation wherever i use required tag. similarly in email tag i need validation of underscore in between hence i need to override the  email tag as well.
if yes, then please share an example. if no, then any alternative.
Thanks in advance.

Thanks @parveenkumar2308
Yes you can override required and email, in fact can override any that is not part of this map by Registering a new validation see here
It's totally up to you, but since you need to register a new validation to override the old one anyway I would register the new email validation, but not override, but instead use a new name like "uemail"(for underscore email) that way you don't lose the functionality of  the regular email too.
and as for the white space in the required tag you could combine required with any of the excludes tags "excludes", "excludesall" or "excludesrune", just so when someone reads your code(assuming someone else will) they can clearly see what's being validated.
You have many options :smile: the docs are pretty thourough if you need some code examples, I'd normally include some buy am out of town with no computer.
Hope this explains what you want, if not after I get back in town can provide some examples.

Thanks Dean Karn.
I have gone through links in your reply. What i have gotten from that is i
need to write a new function like RegisterValidation which makes an entry
in validater map. so, in my case it would be
RequiredWithEmptyCheckValidation and UEmailValidation like you mentioned
that doing it this way i can have both flavours.But, i still didn't get
that where i need to make change in my codebase, where i would define a
util file and i would write the two new validation method or in the API
file itself. also, can you share example, how to define a new one with
exact steps.
I am using validator in my project in below way.i define a struct and give
validation tag against fields and have defined a validation file where i do
validation.
type Person struct{
Id string json:"id" validate:"required,min=36,max=36"
Name string json:"name"
}
in validation.go file
if errs := validate.Struct(obj); errs != nil {
if err := errs.(validator.ValidationErrors)["Person.Id"]; err !=
nil {
if (err.Tag == constant.REQUIRED) {
errMsg = "msg0"
}else{

if (!IsValidStr(obj.Id)) {//here i am checking white spaces in field if
present returning false else true                    errMsg = "msg1"
}*else {
if(err.Tag == constant.MIN){
errMsg =  "msg2"
}else if (err.Tag == constant.MAX) {
errMsg = "msg3"
}
           }
   }

return errMsg

How to implement the new validation method which will cater my if
(!IsValidStr(obj.Id)) {} function itself. Also, I have not used excludes
tags "excludes", "excludesall" or "excludesrune". can you suggest where i
fit this tags or new validation method in my solution without changing much.
Regards,
Parveen Kumar
On Fri, May 20, 2016 at 5:35 PM, Dean Karn<EMAIL_ADDRESS>wrote:

Thanks @parveenkumar2308 https://github.com/parveenkumar2308
Yes you can override required and email, in fact can override any that is
not part of this
https://github.com/go-playground/validator/blob/v8/util.go#L22 map by
Registering a new validation see here
https://github.com/go-playground/validator/blob/v8/validator.go#L270
It's totally up to you, but since you need to register a new validation to
override the old one anyway I would register the new email validation, but
not override, but instead use a new name like "uemail"(for underscore
email) that way you don't lose the functionality of the regular email too.
and as for the white space in the required tag you could combine required
with any of the excludes tags "excludes", "excludesall" or "excludesrune",
just so when someone reads your code(assuming someone else will) they can
clearly see what's being validated.
You have many options 😄 the docs
https://godoc.org/gopkg.in/go-playground/validator.v8 are pretty
thourough if you need some code examples, I'd normally include some buy am
out of town with no computer.
Hope this explains what you want, if not after I get back in town can
provide some examples.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/go-playground/validator/issues/240#issuecomment-220588340

--
Warm Regards,
Parveen Kumar
+91-9890333270

it doesn't really matter where the util file is defined, just need to register the functions.
there are 3 ways you can accomplish what you want with the required, I recommend option 2 or 3 but you may use any you wish.
OPTION 1
package main

import (
	"fmt"
	"reflect"
	"regexp"

	"gopkg.in/go-playground/validator.v8"
)

const (
	uEmailRegexString = "custom email regex"
	uEmail            = "uemail"
	sRequired         = "srequired"
)

var (
	validate    *validator.Validate
	uEmailRegex *regexp.Regexp
)

// Person ...
type Person struct {
	ID    string `json:"id" validate:"srequired,min=36,max=36"`
	Name  string `json:"name"`
	Email string `json:"email" validate:"required,uemail"`
}

func main() {
	uEmailRegex = regexp.MustCompile(uEmailRegexString)

	config := &validator.Config{
		TagName: "validator",
	}

	validate = validator.New(config)
	validate.RegisterValidation(uEmail, IsUEmail)
	validate.RegisterValidation(sRequired, RequiredWithEmptySpace)

	person := &Person{
		ID:    "1",
		Name:  "Joeybloggs",
		Email: "Joey.Bloggs@email.com:",
	}

	errs := validate.Struct(person)

	if errs != nil {
		for k, v := range errs.(validator.ValidationErrors) {
			// Create error messages based on errors returned ...
			fmt.Println(k, v)
		}
	}
}

// IsUEmail ...
func IsUEmail(v *validator.Validate, topStruct reflect.Value, currentStructOrField reflect.Value, field reflect.Value, fieldType reflect.Type, fieldKind reflect.Kind, param string) bool {
	return uEmailRegex.MatchString(field.String())
}

// RequiredWithEmptySpace ...
func RequiredWithEmptySpace(v *validator.Validate, topStruct reflect.Value, currentStructOrField reflect.Value, field reflect.Value, fieldType reflect.Type, fieldKind reflect.Kind, param string) bool {

	var ok bool

	switch fieldKind {
	case reflect.Slice, reflect.Map, reflect.Ptr, reflect.Interface, reflect.Chan, reflect.Func:
		ok = !field.IsNil()
	default:
		ok = field.IsValid() && field.Interface() != reflect.Zero(fieldType).Interface()
	}

	if !ok {
		return ok
	}

	return !validator.ContainsAny(v, topStruct, currentStructOrField, field, fieldType, fieldKind, param)
}

OPTION 2
no need for a custom validation for required, use 'excludesall= '
package main

import (
	"fmt"
	"reflect"
	"regexp"

	"gopkg.in/go-playground/validator.v8"
)

const (
	uEmailRegexString = "custom email regex"
	uEmail            = "uemail"
)

var (
	validate    *validator.Validate
	uEmailRegex *regexp.Regexp
)

// Person ...
type Person struct {
	ID    string `json:"id" validate:"required,excludesall= ,min=36,max=36"` // 'excludesall= ' ensures space is excluded
	Name  string `json:"name"`
	Email string `json:"email" validate:"required,uemail"`
}

func main() {
	uEmailRegex = regexp.MustCompile(uEmailRegexString)

	config := &validator.Config{
		TagName: "validator",
	}

	validate = validator.New(config)
	validate.RegisterValidation(uEmail, IsUEmail)

	person := &Person{
		ID:    "1",
		Name:  "Joeybloggs",
		Email: "Joey.Bloggs@email.com:",
	}

	errs := validate.Struct(person)

	if errs != nil {
		for k, v := range errs.(validator.ValidationErrors) {
			// Create error messages based on errors returned ...
			fmt.Println(k, v)
		}
	}
}

// IsUEmail ...
func IsUEmail(v *validator.Validate, topStruct reflect.Value, currentStructOrField reflect.Value, field reflect.Value, fieldType reflect.Type, fieldKind reflect.Kind, param string) bool {
	return uEmailRegex.MatchString(field.String())
}

OPTION 3
register alias for required + excludesall to avoid some typing
package main

import (
	"fmt"
	"reflect"
	"regexp"

	"gopkg.in/go-playground/validator.v8"
)

const (
	uEmailRegexString = "custom email regex"
	uEmail            = "uemail"
	sRequired         = "srequired"
)

var (
	validate    *validator.Validate
	uEmailRegex *regexp.Regexp
)

// Person ...
type Person struct {
	ID    string `json:"id" validate:"srequired,min=36,max=36"`
	Name  string `json:"name"`
	Email string `json:"email" validate:"required,uemail"`
}

func main() {
	uEmailRegex = regexp.MustCompile(uEmailRegexString)

	config := &validator.Config{
		TagName: "validator",
	}

	validate = validator.New(config)
	validate.RegisterValidation(uEmail, IsUEmail)
	validate.RegisterAliasValidation(sRequired, "required,excludesall= ")

	person := &Person{
		ID:    "1",
		Name:  "Joeybloggs",
		Email: "Joey.Bloggs@email.com:",
	}

	errs := validate.Struct(person)

	if errs != nil {
		for k, v := range errs.(validator.ValidationErrors) {
			// Create error messages based on errors returned ...
			fmt.Println(k, v)
		}
	}
}

// IsUEmail ...
func IsUEmail(v *validator.Validate, topStruct reflect.Value, currentStructOrField reflect.Value, field reflect.Value, fieldType reflect.Type, fieldKind reflect.Kind, param string) bool {
	return uEmailRegex.MatchString(field.String())
}

let me know if this helps @parveenkumar2308, and if you have any more questions just ask.

@joeybloggs i will try the three methods. though i have 1 query with option 2 and 3.
If in the Name field of Person struct, i want to validate for "srequired" tag then , does it allow Parveen Kumar value. because i just want to validate for empty string(if strToCheck == "" || len(strToCheck) == 0 || len(strings.TrimSpace(strToCheck)) == 0 {
return false
}).
not for each white space.

Oh I misunderstood, options 2 & 3 are for any whitespace you are correct.
then option1 would be your best bet with your code changes :)

Hi Dean,
Thanks a lot. It worked like a charm.
Regards,
Parveen Kumar
On Tue, May 24, 2016 at 9:21 AM, Dean Karn<EMAIL_ADDRESS>wrote:

Oh I misunderstood, options 2 & 3 are for any whitespace you are correct.
then option1 would be your best bet with your code changes :)
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/go-playground/validator/issues/240#issuecomment-221159765

--
Warm Regards,
Parveen Kumar
+91-9890333270

No problem, glad it worked for you :)


Request.parse / this._parser not used for browser?

this._parser is set in request-base.js via parse method, but is only used in lib/node/index.js.
It seems like _parseBody in client.js should use it, but it doesn't.
Is this a bug? It appears that maybe this is similar to (or a continuation of) #805?
Also noteworthy and related: buffer(true|false) is not available in the browser.

Neither buffering, nor response body parsing are available in the browser. These things are handled natively by XHR.
In Node parser is expected to listen to events on response stream, but in the browser we don't have a stream (we could #1033)

Ah, so makes sense that buffer(...) would be unavailable within browser.
It still seems like providing a parser via #parse should be used within _parseBody as an override for the global request.parse map?
Current code:
// lib/client.js:375
Response.prototype._parseBody = function(str){
  var parse = request.parse[this.type];
  if (!parse && isJSON(this.type)) {
    parse = request.parse['application/json'];
  }
  return parse && str && (str.length || str instanceof Object)
    ? parse(str)
    : null;
};

What I thought I'd find based on #parse docs:
// lib/client.js:375
Response.prototype._parseBody = function(str){
  var parse = this._parser || request.parse[this.type]; // only changed this line
  if (!parse && isJSON(this.type)) {
    parse = request.parse['application/json'];
  }
  return parse && str && (str.length || str instanceof Object)
    ? parse(str)
    : null;
};

v5.0.0 was released and should fix this issue, see the updated docs


Makefile: Disable buildvcs on go build

Adding this flag removes a build error.
Fixes https://github.com/kiagnose/kubevirt-dpdk-checkup/issues/200

there is no need for this PR right now, the issue was a local one


Error in docs for watchBlockNumber

Is there an existing issue for this?

[X] I have searched the existing issues

Package Version
1.3.9
Current Behavior
Property listen is not an optional argument - contrary to what the docs say
https://github.com/wagmi-dev/wagmi/blob/5e4bc8e26aa9bfc28f654c2d01a38c8d1d7ecc4e/docs/pages/core/actions/watchBlockNumber.en-US.mdx#L52
Expected Behavior
No response
Steps To Reproduce
No response
Link to Minimal Reproducible Example (StackBlitz, CodeSandbox, GitHub repo etc.)
No response
Anything else?
No response

Easy fix if you want credit! PR welcome.

Hey @sunmer and @tmm Can I work on this issue?

Go got it!

Hi @tmm when I try to run a react app after installing wagmi, it gives No wagmi config found. Ensure you have set up a config: https://wagmi.sh/react/config.
Could you please help me?

@SrijanShovit you need to wrap you app with this https://wagmi.sh/react/WagmiConfig

Thanks @glitch-txs

Hey @tmm , I have made a pull request for this issue as PR #2990 . Kindly review. Thanks!!

thanks @tmm 😊


gamesPlayed and setPersona are not work

Like #157
Forum: https://dev.doctormckay.com/topic/1833-did-clientgamesplayed-change/
const SteamUser = require('steam-user')
const steamUser = new SteamUser()

steamUser.logOn({
    account_name: '123',
    password: '123'
})

steamUser.on('loggedOn', details => {
    steamUser.setPersona(SteamUser.EPersonaState.Online)
    steamUser.gamesPlayed(440)
})

This simple code does not work at all. The bot is authorized and details are full of information. But my profile is labeled as "not online" and TF2 does not appear in the games played.
All privacy settings are set to open.

Double-check your privacy settings.

accountName, not account_name.

I had a similar issue, but for me I was using user.setPersona("Online") when I should have used either SteamUser.EPersonaState.Online or 1.


URL was redirecting to 404 page

URL/Link was redirecting to 404 page
Description of the change:
Reason for the change:
Link to original source:

Closes #

hi there! thanks for this PR - we will merge this in early next week. Good catch!

Hello @vk-sourcefuse,Thanks again for the PR! We appreciate your contribution and look forward to                                              continued collaboration. Thanks!                                             Team SendGrid DX


removed problems with empty centers, added json file for ease of use

Bug Description

Some districts like Kottayam(304) have no centers, so payload would be {"centers": []}
Thus checkAvailability to return False, False instead of string as expected,
So if (len(available) > 0) would throw TypeError

Bug Fix

Change if len(available) > 0 to if available

Feature

A settings.json can be provided as an alternative of passing 5 shell arguments.
If settings file is detected, settings would be loaded from that file, else shell arguments would be used.
A settings.example.json file is provided with required schema.

What if we entirely depend on settings.json, ie no command line arguments?
Then an entrypoint script can be created for docker, which would make it it even more easier to run

What if we entirely depend on settings.json, ie no command line arguments?
Then an entrypoint script can be created for docker, which would make it it even more easier to run

What if we entirely depend on settings.json, ie no command line arguments?
Then an entrypoint script can be created for docker, which would make it it even more easier to run

Yea. That's the idea. Use ENTRYPOINT in Dockerfile and let the user put in the arguments in a json file. The only concern is to just keep settings.json and not the sample json file.

Changes

Moved settings.json to settings/settings.json and volume mounted settings to container to persist settings.
Added initial_setup.py which will check settings.json for errors, and ask the users to input data if it finds any

Yea. That's the idea. Use ENTRYPOINT in Dockerfile and let the user put in the arguments in a json file. The only concern is to just keep settings.json and not the sample json file.

I thought you meant user input via shell, that's why i had to add volume.
Letting the user modify settings.json is honestly a better and simpler way

I've made it simple, but settings.json exists in the git index,

If I remove it, git will delete the file in the next commit,
If I don't remove it git will track changes of that file, ie User data could be leaked.

Suggestion

Create a setup script which creates the local settings.json from user input, but without volume the change won't persist, ie every time it executes, user will be prompted for input.
To make it less complex, we could add a setup shell script to run the docker commands, end user won't know the difference.

@rohitanil

What about settings.json stil being in the index @rohitanil? Anyone could accidentally commit their actual credentials to the settings file


NewSpan throw-catch log spam on protected methods

At least, I think that's what's happening.
At debug level, we get the log statement, 'Exception occurred while tyring to find the annotation', with the signature of a protected method.
I can fix it by changing line 82 of SleuthAnnotationUtils to
method.getDeclaringClass().getDeclaredMethod(...

instead of
method.getDeclaringClass().getMethod(...

Also note that the logged message should say 'trying' instead of 'tyring'.
I'll try to submit a pull request when I get some free time.

Pull requests are more than welcome!

I've uploaded a minimal project to illustrate the problem - https://github.com/fisco-unimatic/spring-cloud-sleuth-2072
I now think that it will always find contained spans, but will log a NoSuchMethodException with its stack trace whenever it examines an advised non-public method that is not annotated with both NewSpan and ContinueSpan.
It can be fixed, as I initially said, by calling getDeclaredMethod instead of getMethod, but on close inspection, I think we can just delete  the second attempt to find the annotation.
I think
method.getDeclaringClass().getMethod(method.getName(), method.getParameterTypes())

will always just return method, but I may have misunderstood something, so I'd like a second opinion. Obviously, whoever wrote the method expected that it could sometimes return a different result.
I propose that
static <T extends Annotation> T findAnnotation(Method method, Class<T> clazz) {
	T annotation = AnnotationUtils.findAnnotation(method, clazz);
	if (annotation == null) {
		try {
			annotation = AnnotationUtils.findAnnotation(
					method.getDeclaringClass().getMethod(method.getName(), method.getParameterTypes()), clazz);
		}
		catch (NoSuchMethodException | SecurityException ex) {
			if (log.isDebugEnabled()) {
				log.debug("Exception occurred while tyring to find the annotation", ex);
			}
		}
	}
	return annotation;
}

can just be
static <T extends Annotation> T findAnnotation(Method method, Class<T> clazz) {
	return  AnnotationUtils.findAnnotation(method, clazz);
}

If you change it like this do the test pass?

ping @fisco-unimatic ?


Allow one account for mobile and PC

Allow more than one lightning wallet per account so that you can use one account for mobile and PC.

Maybe add ability to merge one account into an existing account.

tested


What if Cloudflare cookie is expired?

In my experience Bittrex resets CloudFlare's cookie every few hours. Did you test this? May be we need to renew the cookie after disconnection.
Btw // seems like they disabled CloudFlare for now.

I didn't know that it did that, but I can work that in no problem


Add birthdays?

I'm curious if others would find it useful to be able to add someone's birthday to friends and have something like friends suggest inform you if a friend has an upcoming birthday. All opinions welcome!

Once this is added, we could also sort friends by oldest/youngest


Investigate Alternative Approaches for Error Code Object Generation

The current method added by 3b0fdaf0819242cb6866b93f5378fee099edcc96 fixes the issue of attempting to share the same error instance across multiple occurrences of the linting error, but it's extremely repetitive & and there must be a cleaner way to implement it while retaining the same functionality.

Here's an idea:
def define_error(name: str, description: str):
    def init(self, argname: str, lineno: int, col_offset: int):
        Error.__init__(self, name, description)
        self.argname = argname
        self.lineno = lineno
        self.col_offset = col_offset

    globals()[name] = type(name, (Error,), {'__init__': init})

define_error("TYP201", "Missing return type annotation for public function")

Updating the global namespace is optional and probably better to not do so that IDEs can see the definitions exist.

I don't think there's much of an impending need to action this issue so will close for now.


Configure parameters are used inconsistently

We are using this module to authenticate users through firebase. Configuring with the following code works fine on android, but on ios it seems like the webClientId isn't being passed to the sdk and therefore the token returned from the sdk will be different on android and ios.
await GoogleSignin.configure({
    scopes: [ 'https://www.googleapis.com/auth/userinfo.email' ],
    iosClientId: '<iosClientId>',
    webClientId: '<webClientId>',
    offlineAccess: false,
});

In our case this leads to authentication failing on the server for logins from ios.

the webClientId is being set here. Please open a new issue if the problem persists.


Return true when _resultSet is not null

bugfix
https://github.com/apache/pinot/issues/10879
Return true when _resultSet is not null especially when there is no match

@KKcorps

@KKcorps could you help take a look please?

Not introduced in this PR, but per the documentation, seems executeQuery() and execute() shouldn't be allowed in the PreparedStatement. cc @KKcorps

@Jackie-Jiang you are correct but PreparedStatement in our driver is not really doing much. Its just a wrapper on Statement which allows setting queries beforehand.
No optimisation as such is done for the query and we just fill the values and send the whole query again.
So its fine if they are allowed for now.

@zhuangdaz Isn't returning false when result set is empty the correct thing to do?

@KKcorps No, this method should return true if it has successfully run the sql and has non-null resultset(even in this case, there is no row matched. no match != null resultset). Lucas linked an article about this: https://github.com/apache/pinot/issues/10879#issuecomment-1589205560


Simple curl example to fetch Authorize.net transactions

Hello,
Can we fetch the details of individual transactions by submitting the transaction Id from API without using SDK and using simple curl only?
If yes kindly provide an example..
Thanks in advance.

Hi,
The actual API reference for that request is at http://developer.authorize.net/api/reference/index.html#transaction-reporting-get-transaction-details.
There’s a sample request there in XML or JSON format, and all you’d have to do is use curl to post that whole request to the endpoint URL. (Sandbox: https://apitest.authorize.net/xml/v1/request.api or production: https://api.authorize.net/xml/v1/request.api)

On Apr 25, 2017, at 7:17 AM, upstateInc<EMAIL_ADDRESS>wrote:
Hello,
Can we fetch the details of individual transactions by submitting the transaction Id from API without using SDK and using simple curl only?
If yes kindly provide an example..
Thanks in advance.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

With curl on the command line, it would look something like this:
curl -X POST -H "Content-Type: text/xml" -d ' <getTransactionDetailsRequest xmlns="AnetApi/xml/v1/schema/AnetApiSchema.xsd">     <merchantAuthentication>         <name>5KP3u95bQpv</name>         <transactionKey>346HZ32z3fP4hTG2</transactionKey>     </merchantAuthentication>       <transId>12345</transId> </getTransactionDetailsRequest>' "https://apitest.authorize.net/xml/v1/request.api"


Primary key condition parsing [JIRA: RIAK-2299]

When SQL queries are parsed, WHERE clauses involving primary keys are consumed and converted to start/end key pairs (and not passed to the backend filtering).
This works for things like: WHERE time > 100 AND time < 400, but is insufficient for handling combinations of operators like WHERE time > 100 AND time != 400.  These sorts of conditions cannot be converted into simple ranges.
To fix this, the parser should only consume WHERE clauses involving range operators (<, <=, >, >=) but pass other operators as part of the backend filter.  (When I talked to Gordon about this the other day, he stated that this was the original intention, but I have explicitly checked this and the code does not currently pass any conditions on primary keys to the backend)
NB: conversion of primary key conditions to a simple start/end key pair will also fail for OR clauses (ie, WHERE time < 200 OR time > 400).  Under our current model these would have to be converted into two separate range scan queries to the backend.  I have not yet checked that this is handled correctly (but I will!)

Re-assigning this to Andy, since it's really a change in the erlang parser code that needs to happen, and I suspect he's a lot more familiar with it than I am.  Happy to work on it too, but I'll wait for Andy to have a first crack at it.
[posted via JIRA by Erik Leitch]

Working on this currently.
[posted via JIRA by Andy Till]

And as you noted, this is:
partly the unbounded range query problem
ie, we don't support time > 100, therefore can't support time > 100 AND time != 400
which we've decided is definitely not part of TS1.0, in which case we just need to document what's allowed and not allowed for TS1.0
partly error handling
we should throw an error then if we get passed a query like time > 100 AND time != 400
which should definitely be part of TS1.0, and
partly fixable
time > 100 AND time < 400 AND time != 300 could be supported since it's bounded, but we have to pass the last clause to the filter
which could be part of TS1.0
This last bit is tricky, though.  What if someone passes time > 100 AND time < 1000 AND time < 300 AND time > 200?  You'd have to sort through all the time clauses to figure out 1) if you can construct a bounded query, and if so 2) what's the most liberal bound you can construct (ie, time > 100 AND time < 1000 in the above example), and then pass all the rest as part of the filter.  I don't know how much work that is in practice, so do we pursue this now or leave it for a later TS release?
Maybe the best path for now is not to allow these sorts of queries, and make sure we throw meaningful errors if we get an unbounded query?
[~atill] you should comment on this when you've had a chance to scope out the work (or you may already know the answer).
[posted via JIRA by Erik Leitch]

And as you noted, this is:
partly the unbounded range query problem
ie, we don't support time > 100, therefore can't support time > 100 AND time != 400
which we've decided is definitely not part of TS1.0, in which case we just need to document what's allowed and not allowed for TS1.0
partly error handling
we should throw an error then if we get passed a query like time > 100 AND time != 400
which should definitely be part of TS1.0, and
partly fixable
time > 100 AND time < 400 AND time != 300 could be supported since it's bounded, but we have to pass the last clause to the filter
which could be part of TS1.0
This last bit is tricky, though.  What if someone passes time > 100 AND time < 1000 AND time < 300 AND time > 200?  You'd have to sort through all the time clauses to figure out 1) if you can construct a bounded query, and if so 2) what's the most liberal bound you can construct (ie, time > 100 AND time < 1000 in the above example), and then pass all the rest as part of the filter.  I don't know how much work that is in practice, so do we pursue this now or leave it for a later TS release?
Maybe the best path for now is not to allow these sorts of queries, and just make sure we throw meaningful errors if we get an unbounded query?
[~atill] you should comment on this when you've had a chance to scope out the work (or you may already know the answer).
[posted via JIRA by Erik Leitch]

And as you noted, this is:
partly the unbounded range query problem
ie, we don't support time > 100, therefore can't support time > 100 AND time != 400
which we've decided is definitely not part of TS1.0, in which case we just need to document what's allowed and not allowed for TS1.0
partly error handling
we should throw an error then if we get passed a query like time > 100 AND time != 400
which should definitely be part of TS1.0, and
partly fixable
time > 100 AND time < 400 AND time != 300 could be supported since it's bounded, but we have to pass the last clause to the filter
which could be part of TS1.0
This last bit is tricky, though.  What if someone passes time > 100 AND time < 1000 AND time < 300 AND time > 200?  You'd have to sort through all the time clauses to figure out 1) if you can construct a bounded query, and if so 2) what's the most liberal bound you can construct (ie, time > 100 AND time < 1000 in the above example), and then pass all the rest as part of the filter.  I don't know how much work that is in practice, so do we pursue this now or leave it for a later TS release?
Maybe the best path for now is not to allow these sorts of queries, and just make sure we throw meaningful errors if we get an unbounded query?
[~atill] you should comment on this when you've had a chance to scope out the work (or you may already know the answer).
[posted via JIRA by Erik Leitch]

PR https://github.com/basho/riak_kv/pull/1271
[posted via JIRA by Andy Till]


textStartsWith and textContains do not work on iOS

Steps to reproduce

Open "Files" application on iOS with await $.native.openApp(appId: 'com.apple.DocumentsApp');
Tap anything with a Selector passing the text argument and the files appId - it will find the element on the screen. e.g:
await $.native.tap(Selector(text: 'Browse'), appId: 'com.apple.DocumentsApp');
Tap anything with a Selector passing the textStartsWith argument and the files appId - it will not find the element. e.g:
await $.native.tap(Selector(textStartsWith: 'Browse'), appId: 'com.apple.DocumentsApp');

My assumption reading the logs is that it ignores the textStartsWith parameter and only takes the text parameter into consideration.
I've even tried it with textContains and it doesn't work as well.
It works perfectly on Android.
Logs

Logs
2023-07-13 17:57:51.765935-0300 RunnerUITests-Runner[48509:3435770] PatrolServer: INFO: tapping on view with text "" in app com.apple.DocumentsApp...
2023-07-13 17:57:51.766523-0300 RunnerUITests-Runner[48509:3435770] PatrolServer: INFO: waiting for existence of view with text ""
    t =    20.04s Waiting 10.0s for Any to exist
    t =    21.07s     Checking `Expect predicate `exists == 1` for object Any`
    t =    21.07s         Checking existence of `Any`
    t =    21.11s         Capturing element debug description
    t =    22.15s     Checking `Expect predicate `exists == 1` for object Any`
    t =    22.15s         Checking existence of `Any`
    t =    22.17s         Capturing element debug description
    t =    23.21s     Checking `Expect predicate `exists == 1` for object Any`
    t =    23.21s         Checking existence of `Any`
    t =    23.25s         Capturing element debug description
    t =    24.28s     Checking `Expect predicate `exists == 1` for object Any`
    t =    24.28s         Checking existence of `Any`
    t =    24.33s         Capturing element debug description
    t =    25.36s     Checking `Expect predicate `exists == 1` for object Any`
    t =    25.36s         Checking existence of `Any`
    t =    25.39s         Capturing element debug description
    t =    26.42s     Checking `Expect predicate `exists == 1` for object Any`
    t =    26.42s         Checking existence of `Any`
    t =    26.46s         Capturing element debug description
    t =    27.50s     Checking `Expect predicate `exists == 1` for object Any`
    t =    27.50s         Checking existence of `Any`
    t =    27.55s         Capturing element debug description
    t =    28.58s     Checking `Expect predicate `exists == 1` for object Any`
    t =    28.58s         Checking existence of `Any`
    t =    28.62s         Capturing element debug description
    t =    29.66s     Checking `Expect predicate `exists == 1` for object Any`
    t =    29.66s         Checking existence of `Any`
    t =    29.70s         Capturing element debug description
    t =    30.04s Checking existence of `Any`
2023-07-13 17:58:01.810559-0300 RunnerUITests-Runner[48509:3435891] PatrolServer: ERROR: view with text "" in app "com.apple.DocumentsApp" doesn't exist

Patrol version
patrol: 2.0.7
Patrol Doctor output

Patrol Doctor output
% patrol doctor
You're using Patrol CLI 2.0, which has breaking changes.
Read the migration guide at https://patrol.leancode.co/v2.
Disable this warning by setting the PATROL_MIGRATED environment variable.

Patrol CLI version: 2.0.4
Program adb found in /Users/cesarcastro/Library/Android/sdk/platform-tools/adb
Env var $ANDROID_HOME is not set
Program xcodebuild found in /usr/bin/xcodebuild
Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller

Flutter Doctor output

Flutter Doctor output
% flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.7.12, on macOS 13.4.1 22F82 darwin-arm64, locale pt-BR)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.1)
[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
[✓] Chrome - develop for the web
[!] Android Studio
    ✗ Android Studio not found at ~/Library/Application\ Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/201.7199119/Android\ Studio.app/Contents
[✓] Android Studio (version 2022.2)
[✓] IntelliJ IDEA Community Edition (version 2023.1.3)
[✓] VS Code (version 1.79.2)
[✓] Connected device (3 available)
[✓] HTTP Host Availability

! Doctor found issues in 1 category.

Thank you for reporting it - it's just not implemented on iOS. Sorry about that. Actually only Selector.text has the effect on iOS.
We should get it fixed so things like Selector.textStartsWith and Selector.textContains work on iOS as well. Some (roughly related) work on that has been started in #1054.


[Question]: Empty Response after query_engine.query

Question Validation

[x] I have searched both the documentation and discord for an answer.

Question
I am trying to read the content of website and index it using llama_index but after I perform query_engine.query(question) I get a empty response.I have latest version of llama_index installed.(0.8.31)
When I saw the index_store.json file its content was:
{"index_store/data": {"some_string": {"type": "vector_store", "data": "{"index_id": "some_string", "summary": null, "nodes_dict": {}, "doc_id_dict": {}, "embeddings_dict": {}}"}}}
Below is the code:
def document_loader():
web_links = ["https://www.databricks.com/","https://help.databricks.com","https://databricks.com/try-databricks","https://help.databricks.com/s/","https://docs.databricks.com"]
loader = WebBaseLoader(web_links)
documents = loader.load()
docs=[Document(page_content=doc.page_content,metadata={"source":doc.metadata["source"]}) for doc in documents]
model_name = "sentence-transformers/all-mpnet-base-v2"
model_kwargs = {"device": "cuda"}
embeddings =LangchainEmbedding(
HuggingFaceEmbeddings(model_name=model_name, model_kwargs=model_kwargs)
)
service_context = ServiceContext.from_defaults(
chunk_size=2048,
llm=llm,
embed_model=embeddings
)
set_global_service_context(service_context)
index=GPTVectorStoreIndex.from_documents(docs)
index.storage_context.persist("db/naval_index")
storage_context = StorageContext.from_defaults(persist_dir="db/naval_index")
new_index = load_index_from_storage(storage_context)
new_query_engine = new_index.as_query_engine()
new_query_engine.query(question)
Is it the problem with llama_index or my approach is incorrect?

When you load the index, you should pass the service context back in
new_index = load_index_from_storage(storage_context, service_context=service_context)

Also, it looks like you are using the langchain document class? Or you set the wrong kwargs. Should be
docs=[Document(text=doc.page_content, metadata={"source":doc.metadata["source"]}) for doc in documents]

Thankyou for the response @logan-markewich. Again now I removed the manual Document creation code and replaced the web loader from Langchain to BeautifulSoupWebReader in llama_index.
Below is the index.json file content:
{"index_store/data": {"some_string": {"type": "vector_store", "data": "{"index_id": "some_string", "summary": null, "nodes_dict": {"2e48e410-a236-4f61-a282-e94429cb9bb9": "2e48e410-a236-4f61-a282-e94429cb9bb9"}, "doc_id_dict": {}, "embeddings_dict": {}}"}}}
But still doc_id_dict and embeddings_dict are empty.
Below is the code:
from llama_index.embeddings import LangchainEmbedding
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index import set_global_service_context
from llama_index import ServiceContext
Import deps to load documents
from llama_index import VectorStoreIndex, download_loader, GPTVectorStoreIndex
from llama_index import StorageContext, load_index_from_storage
def document_loader():
web_links = ["https://www.databricks.com/"]
BeautifulSoupWebReader = download_loader("BeautifulSoupWebReader")
loader = BeautifulSoupWebReader()
documents = loader.load_data(urls=web_links)
docs=[Document(text=doc.page_content,metadata={"source":doc.metadata["source"]}) for doc in documents]
model_name = "sentence-transformers/all-mpnet-base-v2"
model_kwargs = {"device": "cuda"}
embeddings =LangchainEmbedding(
HuggingFaceEmbeddings(model_name=model_name, model_kwargs=model_kwargs)
)
service_context = ServiceContext.from_defaults(
chunk_size=2048,
llm=llm,
embed_model=embeddings
)
# And set the service context
set_global_service_context(service_context)
index=GPTVectorStoreIndex.from_documents(documents)
index.storage_context.persist("db/naval_index")
storage_context = StorageContext.from_defaults(persist_dir="db/naval_index")
new_index = load_index_from_storage(storage_context,service_context=service_context)
new_query_engine = new_index.as_query_engine()
response=new_query_engine.query("What is Databricks")
print(response)
Output:

Output now is blank atleast earlier it was printing Empty Response!!

Hmm I suspect if you print(response.source_nodes) it will show the retrieved nodes properly
Seems like maybe an issue with the LLM? Which LLM are you using, or how is it setup?


Remove unnecessary line break in Counting Cards

Tested locally. No issue reference.
Before:

After:

:+1:


Grouping on complex columns aka unifying GroupBy strategies

Description
User Impact
Once the core functionality is in, users can pass complex types as dimensions to the group by queries. For example:
SELECT nested_col1, count(*) FROM foo GROUP BY nested_col1

Making grouping strategies type agnostic
The grouping engine requires a way to address a dimension and store a fixed-width representation of that dimension on a buffer. This can vary for different types, for example, longs can be represented as is, while strings require a dictionary (runtime, or precomputed) so that a variable length string can be addressed by a fixed width dictionary id.
Currently, we have a separate strategy for each type. For enabling grouping on complex columns (at the engine level), we need these strategies to be type-agnostic, so that they can work with the supported complex types, without knowing the types. We can also unify the pre-existing types so that the grouping strategies are classified based on the work (and dimensions) that they handle, rather than being specialized for each type. These are the following classifications:

Fixed width types - These include numeric primitives. In the future, we can optimize fixed-width complex types, like IPv4 and Geo types to use this strategy instead, with some hints from the type strategy, that the objects can be represented in fixed-width columns. Such types can be serialized into their representations without being mapped to a dictionary, as they are fixed widths.

Variable width types with prebuilt dictionaries - These include string types with prebuilt dictionaries. In the future, once we have a way of extracting array dictionaries from the "array columns selectors", we can have array types to use this strategy as well. Such types can be represented using their prebuilt dictionaryId

Variable width types without prebuilt dictionaries - These include any variable type without a prebuilt dictionary, including complex types. We need to build a dictionary + reverse dictionary corresponding to each key and represent the key using the dictionaryId. This also requires a way to estimate the size of the dictionaries and address the key on the dictionary - which requires some interface changes.

Implementing a hashing strategy for complex types
Group by on complex columns requires that the complex types are addressable from a reverse dictionary.
Currently, all the reverse dictionaries are of pre-known types, and hence the dictionary implementations themselves (hash maps) supply the hashCode and equals required to address the object on the dictionary. Dictionaries can be of two types, and we lay down the ins and outs of addressing an object on both the types:
Associative array, i.e. HashMap (Object2IntOpenHashMap)
This is currently used throughout the code.
For any type to be added to a map, it requires the correct implementation of two default methods of Object class

hashCode() : It is required to associate a position in the hash table with the object. Same objects must have the same hashCodes, while different objects can have the same hashcodes as well, which leads to the following point.

equals(): It is required for disambiguating the objects in case of collisions, i.e. when they have the same hashCode.

Sorted Set (Object2IntRBTreeMap/ Object2IntAvlTreeMap)
This would be a change in the existing dictionary-building strategy, therefore it will affect the performance of array types and string types without a prebuilt dictionary. Hence this change is benchmarked.
The benefit of using this would be that we can reuse the comparators for addressing the objects in the arrays.
However, this affects the performance of the grouping engine, therefore this design is dropped (check the benchmarks section).
Grouping on non useful types
Should grouping on types like HLLs, Pairs etc (used for earliest/latest), be allowed or not? There isn’t any practical use case for supporting these types, however, they can fall under the catch-all umbrella, where we serde the type, and compare the byte representation. Therefore, the major question is to or not to?
Currently, grouping on these types is disallowed, because they are often counter-intuitive to what the users want to measure. For example, the user might want to group on the finalized estimated count, and rather they are grouping on the sketch itself.
Interface changes
Following interface changes have been proposed to implement the stuff above:
TypeStrategy.java

public interface TypeStrategy<T> extends Comparator<Object>, Hash.Strategy<T>
{

  ....
  // Whether the type is groupable
  default boolean groupable()
  {
    return false;
  }

  // Implement for hashing complex dimensions
  @Override
  default int hashCode(T o)
  {
    throw DruidException.defensive("Not implemented. Check groupable() first");
  }

  // Implement for hashing complex dimensions
  @Override
  default boolean equals(T a, T b)
  {
    throw DruidException.defensive("Not implemented. Check groupable() first");
  }
  .......
}

Benchmarks
The performance of the current and the changes made using this patch are equivalent, while if we use the sorted sets, it degrades, and hence that approach is rejected

master i.e. current

Benchmark                                  (groupingDimension)  Mode  Cnt      Score       Error  Units
SqlGroupByBenchmark.querySql    stringArray-Sequential-100_000  avgt    5    946.754 ±  198.785  ms/op
SqlGroupByBenchmark.querySql  stringArray-Sequential-3_000_000  avgt    5  41515.654 ± 4081.663  ms/op
SqlGroupByBenchmark.querySql        stringArray-ZipF-1_000_000  avgt    5    435.283 ±   21.554  ms/op
SqlGroupByBenchmark.querySql     stringArray-Uniform-1_000_000  avgt    5   7669.069 ±  629.368  ms/op
SqlGroupByBenchmark.querySql      longArray-Sequential-100_000  avgt    5    820.420 ±   58.626  ms/op
SqlGroupByBenchmark.querySql    longArray-Sequential-3_000_000  avgt    5  43184.769 ± 6502.256  ms/op
SqlGroupByBenchmark.querySql          longArray-ZipF-1_000_000  avgt    5    319.639 ±   17.426  ms/op
SqlGroupByBenchmark.querySql       longArray-Uniform-1_000_000  avgt    5   6242.893 ±  279.628  ms/op

this branch + using associative array i.e. post this patch

Benchmark                                  (groupingDimension)  Mode  Cnt      Score       Error  Units
SqlGroupByBenchmark.querySql    stringArray-Sequential-100_000  avgt    5    852.758 ±   136.371  ms/op
SqlGroupByBenchmark.querySql  stringArray-Sequential-3_000_000  avgt    5  41204.688 ±  8408.370  ms/op
SqlGroupByBenchmark.querySql        stringArray-ZipF-1_000_000  avgt    5    390.198 ±    12.634  ms/op
SqlGroupByBenchmark.querySql     stringArray-Uniform-1_000_000  avgt    5   5509.581 ±   451.345  ms/op
SqlGroupByBenchmark.querySql      longArray-Sequential-100_000  avgt    5    660.656 ±    57.070  ms/op
SqlGroupByBenchmark.querySql    longArray-Sequential-3_000_000  avgt    5  47083.740 ± 20552.695  ms/op
SqlGroupByBenchmark.querySql          longArray-ZipF-1_000_000  avgt    5    300.093 ±    38.598  ms/op
SqlGroupByBenchmark.querySql       longArray-Uniform-1_000_000  avgt    5   5725.369 ±   725.885  ms/op

*this branch + using sorted sets i.e. discarded dictionary building approach
Benchmark                                  (groupingDimension)  Mode  Cnt      Score       Error  Units
SqlGroupByBenchmark.querySql    stringArray-Sequential-100_000  avgt    5   1221.906 ±   64.073  ms/op
SqlGroupByBenchmark.querySql  stringArray-Sequential-3_000_000  avgt    5  51887.836 ± 2145.206  ms/op
SqlGroupByBenchmark.querySql        stringArray-ZipF-1_000_000  avgt    5   1013.708 ±  110.974  ms/op
SqlGroupByBenchmark.querySql     stringArray-Uniform-1_000_000  avgt    5   9127.126 ±  183.487  ms/op
SqlGroupByBenchmark.querySql      longArray-Sequential-100_000  avgt    5    819.836 ±   26.940  ms/op
SqlGroupByBenchmark.querySql    longArray-Sequential-3_000_000  avgt    5  41988.609 ± 3683.729  ms/op
SqlGroupByBenchmark.querySql          longArray-ZipF-1_000_000  avgt    5    695.684 ±   21.028  ms/op
SqlGroupByBenchmark.querySql       longArray-Uniform-1_000_000  avgt    5   7877.897 ± 1755.698  ms/op
SqlGroupByBenchmark.querySql         nested-Sequential-100_000  avgt    5     77.833 ±    7.598  ms/op
SqlGroupByBenchmark.querySql       nested-Sequential-3_000_000  avgt    5    157.890 ±    7.128  ms/op
SqlGroupByBenchmark.querySql             nested-ZipF-1_000_000  avgt    5    103.281 ±    8.962  ms/op

Future work
Once the core functionality is up, we can have the following previously alluded to improvements for faster grouping of specific types:

Implement a group by on complex columns and arrays for vectorized query engine. This can be done in a follow-up PR, without requiring an interface change.

Once there’s a way to extract the dictionaries from array selectors, we can reuse those dictionaries, i.e. strategy (2), rather than rebuilding the dictionaries. This requires a new selector class/interface like ArrayColumnValueSelector, akin to  DimensionSelector which would be a massive code change across Druid’s codebase

We can hint to the grouping engine about a few complex types being fixed width beforehand - these include the IP types. This would prevent the need for having a dictionary associated with the type, and instead, they can use strategy (1) to directly serialize the type onto the buffer. This will require adding methods to the public interface, which I wanna defer to the future patches

Release note

Key changed/added classes in this PR

MyFoo
OurBar
TheirBaz

This PR has:

[ ] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

I am cleaning up the dead todos, and adding a few tests soon! Should be good to go then.


Render error after adding Vercel or Netlify configuration

After npx astro add vercel or npx astro add vercel upon opening the project pages, that is http://localhost:4321/projects/project-1/ or others, this error is encountered:
Cannot read properties of undefined (reading 'render')
In projects/[...slug].astro:13:33:
TypeError: Cannot read properties of undefined (reading 'render')
    at /home/me/squealing-sphere/src/pages/projects/[...slug].astro:13:33
    at [...slug] (/home/me/squealing-sphere/node_modules/astro/dist/runtime/server/astro-component.js:16:12)
    at callComponentAsTemplateResultOrResponse (file:///home/me/squealing-sphere/node_modules/astro/dist/runtime/server/render/astro/render.js:90:31)
    at renderToAsyncIterable (file:///home/me/squealing-sphere/node_modules/astro/dist/runtime/server/render/astro/render.js:132:32)
    at renderPage (file:///home/me/squealing-sphere/node_modules/astro/dist/runtime/server/render/page.js:31:30)
    at lastNext (file:///home/me/squealing-sphere/node_modules/astro/dist/core/render-context.js:135:31)
    at async callMiddleware (file:///home/me/squealing-sphere/node_modules/astro/dist/core/middleware/callMiddleware.js:21:10)
    at async RenderContext.render (file:///home/me/squealing-sphere/node_modules/astro/dist/core/render-context.js:163:22)
    at async handleRoute (file:///home/me/squealing-sphere/node_modules/astro/dist/vite-plugin-astro-server/route.js:156:16)
    at async run (file:///home/me/squealing-sphere/node_modules/astro/dist/vite-plugin-astro-server/request.js:40:14)

It occurs when the astro.config.mjs is modified to:
either
import { defineConfig } from 'astro/config';
import vercel from "@astrojs/vercel/serverless";

// https://astro.build/config
export default defineConfig({
  output: "server",
  adapter: vercel()
} // Your original configuration here
);

or
import { defineConfig } from 'astro/config';
import netlify from "@astrojs/netlify";

// https://astro.build/config
export default defineConfig({
  output: "server",
  adapter: netlify()
} // Your original configuration here
);

It is fixed when the astro.config.mjs is modified back to:
import { defineConfig } from 'astro/config';

export default defineConfig({
  // Your original configuration here
});

Subject : Resolving "Cannot read properties of undefined (reading 'render')" in a Server-Side Astro configuration.
Problem:
when you use the following configuration in astro.config.mjs:
export default defineConfig({
  output: "server",
  adapter: vercel()
});

you are going to encounter the error Cannot read properties of undefined (reading 'render') in your dynamic page components. This error occurs due to the change in you're execution context to server-side rendering.
Solution:
To resolve this error, you just have to explicitly set the prerender property to true in the dynamic page component to solve the issue. (in you're case the page is [...slug].astro)
export const prerender = true;

This will indicate to Astro to pre-render the page during the build process and result into generating a static HTML that can be served directly by the server.
Explanation:
When using the output: "server" configuration, Astro renders pages on the server-side. This introduce differences in how components and date are accessed compared to client-side rendering. By pre-rendering the page, you ensure that the component is rendered and all necessary data is available before the page is srved to the client.

Apologies for the delayed response—I've been busy with moving. @iCaran I know this is coming late, but the fix has now been pushed. Thanks for bringing it to my attention, as I don’t use Vercel for hosting this project. @frex-is I really appreciate your solution and assistance with the issue—super helpful of you!


Shape mismatch when using lstsq_simulate with TF backend

I came across your paper on the arxiv and wanted to test out some of the examples.  I ran into an issue when running lstsq_simulate with the TF backend:
ValueError: Shape must be rank 4 but is rank 3 for '{{node concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](zeros, PartitionedCall_1, concat/axis)' with input shapes: [0,60,60,1], [60,60,1], [].

Looking around the source code I think I tracked down the issue to these lines:
https://github.com/giga-lens/gigalens/blob/master/src/gigalens/tf/simulator.py#L119-L121
Specifically, there is a shape mismatch between img and tmp.
I took a look at some of the other branches and this seems to be fixed on the tf-demo branch:
https://github.com/giga-lens/gigalens/blob/tf-demo-update/gigalens/tf/simulator.py#L119-L123
But I am reluctant to switch to that branch as it seems to be out of date with the master branch.  I figured I would flag it up as this bug appears to be present in the package's current pypi release.

I just tested the JAX backend and found the same issue there as well:
TypeError: Cannot concatenate arrays with different numbers of dimensions: got (0, 120, 120, 1), (120, 120, 1).
Looks to be these lines: https://github.com/giga-lens/gigalens/blob/master/src/gigalens/jax/simulator.py#L110-L114


Plugin UI

Hello,
I was going to work on building UI views but it looks like you already have a plugin for this?  I didn't see it in the plugin market though?
Thx

@billiemead
Plugin UI I plan to release in the next version, it will be integrated in an admin backend

@billiemead
Plugin UI I plan to release in the next version, it will be integrated in an admin backend

Ok! Let me know if there are any tasks you need help with. I am adding en and zh-CN languages to laravel-plugin-market right now but happy to help if you need any. Cheers! Billie

@billiemead
👌，Thank you so much.


Add very simple dashboard customizations

Are you planning on adding kind of metrics? Graphs, etc...?

The idea is to have textual widgets and some very simple graph widgets. Currently I have an early prototype of how this is going to work. I think it will be a great addition to kaffy.
Do you have a specific need?

This is done and released in v0.5.0
This will be improved in later versions, but the basis is there.


Authority API and URL & QR code device linking

Allows for authority transfer between multiple clients at the same time.
const volume = await program.account.volume()

// Provider
await program.authority.provide([
  odd.authority.account,
  odd.authority.fileSystem.rootAccess(volume.did)
])

// Requestor
await program.authority.request(
  odd.authority.account
)

await program.authority.request(
  odd.authority.fileSystem.rootAccess(volume.did)
)

Demo
https://github.com/oddsdk/ts-odd/assets/296665/c1ccff06-87cd-4586-a874-20b2ffce5edf
⚠ There's an issue with the websocket server causing connection issues. When you close the connection and reopen it, the messages aren't received by the other end.
How it works

One or more clients call  program.authority.provide() with authorisation queries. Queries are a way to select various UCANs and access keys. Above we have three queries: (1) query for the account capabilities (account component determines what these will be), (2 & 3) query for read & write access to the public & private roots of the file system
You get a URL as the result from .provide(). This URL can be converted into a QR code.
The URL is opened elsewhere.
Whenever program.authority.request() is called, it checks for the query params associated with the authority component implementation. authority[challenge] and authority[publicKey] specifically.
.request() detects the query params, establishes the channel via the channel component, making a websocket connection to the producer.
The second part of the handshake is done, confirming various cryptographic values. Producer tells requestor that everything looks in order.
Requestor sends over their query.
Producer compares the query of the requestor with their own. The received query must be equal of contained by configured provision query (eg. if you share the path private/subdir/, the requestor cannot ask for acces to private/)
Producer looks up the access keys and UCANs, then sends them over the channel encrypted.
Requestor decrypts keys and UCANs.
Authority transfer completed ✅

Additional notes

This does not ping the producer until they're online. Because that concept doesn't make sense here seeing that every session will have unique key-pair and thus unique URL/QR-code. If you've opened a link and closed the producer, the session becomes useless. Instead you open the producer again and use the new URL/QR-code.
You can provide and request at the same time.

Other changes

Adjusted Inventory to also work with access keys.
file system authority queries now require a DID parameter.
Fixes some file system issues (rs-wnfs has been updated too)
Moved all dependencies that are only used by components to peer dependencies (as to separate them from the sdk core)

Test Plan

Register an account
Load a file system (may also be done before registering)
Make a change in the file system
Provide authority (can be done earlier too)
Request authority using the code above
Success! Hopefully 🙈

Superseded by #572


prnt.sc

::: /etc/pihole/list.41.hosts.ubuntu101.co.za.domains (1 results)
<IP_ADDRESS> prnt.sc
need your opinion on this. yes? no? maybe?

Looks fine to me, adding to false-positives and fixed in next commit shortly. 👍


Tooling Check

For more information see the link below.
https://github.com/ballerina-platform/ballerina-standard-library/blob/main/docs/library-development-process.md#tooling-check

Following cases were tested related to tooling tests.

[x] Code formatting
[x] Code completion
[x] Symbol information on hover
[x] Go to definition

Added compiler plugin validation - https://github.com/ballerina-platform/ballerina-standard-library/issues/1514
Added code generation - https://github.com/ballerina-platform/ballerina-standard-library/issues/2704


Add tipsForRequestingData as as standalone page

And showcase this content so that it helps others!

Done! https://github.com/atlantichabitatmodels/data/commit/eaff13c41450b5160992d33d5eefcc2079669e1f


Build js env

2 changes in this PR:

Bumping js packages as I couldn't install sharp
Add docker support to run everything locally without need for any extra dev setup

Please review changes @ines, hope they are helpful

Thanks a lot for your work on this, I definitely think the Makefile and Docker setup are going to be useful to others 👍 Also thanks for updating the docs!
Just testesd this package.json setup and dependency configuration locally by installing everything from scratch and it works fine for me.
I don't know Docker very well, so I can't really review that part of it. Is there a way we can fold that config.env file into the Makefile? Or is that a Docker convention and required? I'm always trying to reduce the number of files and configs etc.


tedge-mapper-c8y does not shutdown cleanly if JWT retriever is retrying

Describe the bug

When the tedge-mapper-c8y is trying to get a Cumulocity IoT JWT (token) but is having issues, it waits 60 seconds and tries again. It looks like the "wait 60 seconds" logic (or the JWT retriever actor) does not subscribe to the shutdown signal. This is visible because when restarting the service via systemd (e.g. `systemctl restart tedge-mapper-c8y), it takes the service ~30 seconds to restart.
The actor runtime at least handles this "unresponsiveness" as shown be the log snippet below, however the tedge-mapper-c8y in normal scenarios (e.g. when the JWT token retry logic is running) the Runtime: Timeout errors should not occur.
2023-05-11T06:28:17.213441502Z ERROR Runtime: Timeout waiting for all actors to shutdown

To Reproduce

Assuming you have an already running thin-edge.io setup (including a connection to Cumulocity IoT):

Disconnect the network (or alternatively, edit the /etc/tedge/mosquitto-conf/c8y-bridge.conf bridge settings to create invalid settings, though then you will need to restart mosquitto using systemctl restart mosquitto)
Restart tedge-mapper-c8y using systemctl restart tedge-mapper-c8y (if the restart takes < 2 seconds, then try restarting the service again)

Expected behavior

The JWT retriever logic should be interrupted when receiving the shutdown signal from the actor runtime.

Screenshots

The journalctl logs from the tedge-mapper-c8y service are shown below. The snippet shows the lead up to executing the systemctl restart tedge-mapper-c8y command, and after the service restarts
May 11 06:26:30 d1d433801948 tedge-mapper[87]: 2023-05-11T06:26:30.844659577Z  INFO C8Y-REST => JWT: send ()
May 11 06:26:30 d1d433801948 tedge-mapper[87]: 2023-05-11T06:26:30.844693577Z  INFO C8YJwtRetriever: recv Some((0, ()))
May 11 06:26:40 d1d433801948 tedge-mapper[87]: 2023-05-11T06:26:40.84598579Z  INFO C8YJwtRetriever: send (0, Err(NoResponse))
May 11 06:26:40 d1d433801948 tedge-mapper[87]: 2023-05-11T06:26:40.846059123Z  INFO C8Y-REST => JWT: recv Some(Err(NoResponse))
May 11 06:26:40 d1d433801948 tedge-mapper[87]: 2023-05-11T06:26:40.846067915Z ERROR c8y_http_proxy::actor: An error occurred while retrieving internal Id, operation will retry in 60 seconds
May 11 06:26:40 d1d433801948 tedge-mapper[87]:  Error: CustomError("JWT token not available")
May 11 06:27:17 d1d433801948 systemd[1]: Stopping tedge-mapper-c8y converts Thin Edge JSON measurements to Cumulocity JSON format....
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211797724Z  INFO Signal-Handler: send Shutdown
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211839224Z  INFO Runtime: Shutting down
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211851224Z  INFO Runtime: Successfully sent shutdown request to MQTT-2
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211866724Z  INFO Runtime: Successfully sent shutdown request to CumulocityMapper-8
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211870349Z  INFO Runtime: Successfully sent shutdown request to MQTT-9
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.21187339Z  INFO Runtime: Successfully sent shutdown request to Signal-Handler-0
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211875932Z  INFO Runtime: Successfully sent shutdown request to C8YJwtRetriever-3
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211888307Z  INFO Runtime: Successfully sent shutdown request to C8Y-REST-5
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211903432Z  INFO Runtime: Successfully sent shutdown request to HealthMonitorActor-1
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211907599Z  INFO Runtime: Successfully sent shutdown request to Timer-7
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211910474Z  INFO Runtime: Successfully sent shutdown request to FsWatcher-6
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211912765Z  INFO Runtime: Successfully sent shutdown request to HTTP-4
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.211927307Z  INFO HTTP: recv None
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212101224Z  INFO Runtime: Actor has finished: HTTP-4
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212158849Z  INFO MQTT: recv Err(Shutdown)
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212181807Z  INFO Runtime: Actor has finished: MQTT-9
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212195474Z  INFO tedge-mapper-c8y: recv None
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212221057Z ERROR Runtime: Actor FsWatcher-6 has finished unsuccessfully: ChannelError(ReceiveError)
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.21226839Z  INFO Signal-Handler: recv None
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212280224Z  INFO C8YJwtRetriever: recv None
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212288474Z  INFO Runtime: Actor has finished: C8YJwtRetriever-3
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212344474Z  INFO Runtime: Actor has finished: HealthMonitorActor-1
May 11 06:27:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:17.212381807Z  INFO Runtime: Actor has finished: Signal-Handler-0
May 11 06:27:40 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:40.846804943Z  INFO C8Y-REST => JWT: send ()
May 11 06:27:40 d1d433801948 tedge-mapper[87]: 2023-05-11T06:27:40.846840902Z ERROR c8y_http_proxy::actor: An error occurred while retrieving internal Id, operation will retry in 60 seconds
May 11 06:27:40 d1d433801948 tedge-mapper[87]:  Error: FromChannel(SendError(SendError { kind: Disconnected }))
May 11 06:28:17 d1d433801948 tedge-mapper[87]: 2023-05-11T06:28:17.213441502Z ERROR Runtime: Timeout waiting for all actors to shutdown
May 11 06:28:17 d1d433801948 systemd[1]: tedge-mapper-c8y.service: Succeeded.
May 11 06:28:17 d1d433801948 systemd[1]: Stopped tedge-mapper-c8y converts Thin Edge JSON measurements to Cumulocity JSON format..
May 11 06:28:17 d1d433801948 systemd[1]: Started tedge-mapper-c8y converts Thin Edge JSON measurements to Cumulocity JSON format..
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.223618836Z  INFO flockfile::unix: Lockfile created "/run/lock/tedge-mapper-c8y.lock"
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.223785419Z  INFO Runtime: Started
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.223805544Z  INFO Runtime: Running Signal-Handler-0
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.223815044Z  INFO Runtime: Running HealthMonitorActor-1
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.223856752Z  INFO tedge-mapper-c8y: send Message { topic: Topic { name: "tedge/health/tedge-mapper-c8y" }, payload: "{"pid":7211,"status":"up","time":1683786497}", qos: AtLeastOnce, retain: true }
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227124086Z  INFO Runtime: Running MQTT-2
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227154544Z  INFO Runtime: Running C8YJwtRetriever-3
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227160127Z  INFO Runtime: Running HTTP-4
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227184586Z  INFO Runtime: Running C8Y-REST-5
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227191627Z  INFO Runtime: Running FsWatcher-6
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227197002Z  INFO Runtime: Running Timer-7
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227477002Z  INFO C8Y-REST: start initialisation
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227520961Z  INFO C8Y-REST => JWT: send ()
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.227540169Z  INFO C8YJwtRetriever: recv Some((0, ()))
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.228134002Z  INFO Runtime: Running CumulocityMapper-8
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.228144336Z  INFO Runtime: Running MQTT-9
May 11 06:28:17 d1d433801948 tedge-mapper[7211]: 2023-05-11T06:28:17.229476294Z  INFO MQTT: recv Ok(Some(Message { topic: Topic { name: "tedge/health/tedge-mapper-c8y" }, payload: "{"pid":7211,"status":"up","time":1683786497}", qos: AtLeastOnce, retain: true }))

Environment (please complete the following information):

OS [incl. version]: debian 11
Hardware [incl. revision]: docker engine (aarch64)
System-Architecture [e.g. result of "uname -a"]: aarch64
thin-edge.io version [e.g. 0.1.0]: 0.10.0-223-g62e38953

Additional context

Very closely related to a similar issue #1934

So, this is in fact more subtle and this is due to an old friend.
The c8y mapper needs to know the internal id of the device and loop till this internal id is fetched from Cumulocity. This internal id is required only for some operation (notably to post the software list).
Currently Runtime::Shutdown requests are ignored during this loop.
This can be fixed using different approaches:

(simplest) Make this init loop interruptible by shutdown requests.
Trigger the initialization of the internal id only when the c8y bridge is up.
Store on disk the internal id to avoid having to request it each time the mapper is launched.

@reubenmiller what would be your preference?

Store on disk the internal id to avoid having to request it each time the mapper is launched.

The last one please. Though in addition there should be some invalidation logic when using the internal-id code. For example if any of the Cumulocity HTTP requests which use the internal-id return; 403 or 404 HTTP status codes, then the internal id should be discarded and the internal-id should be looked up again using the external id.
The following HTTP response status codes can occur for the following reasons:
404 (Not Found): Occurs if the managed object is deleted in Cumulocity (this actually occurs more often than you think, especially in development).
403  (Permission denied): The device is no longer the owner of the managed object, this can happen if someone has been changing the owner of the managed object, or moved the external id from one managed object to another (migration scenario).

Tested and it is ok


Problem with Resize event

Hi @russellgoldenberg
Thanks a lot for that great piece of code.
I've encountered an issue trying to setup a resize function. I've tried to throttle the resize event, or debounce it like i've did there : scrollama-problem-resize
The problem is that, on some resize events, two steps elements become active when only one is supposed to get the class="is-active" attached to it. Reload the page or scrolling to the next step solve the issue, so there might be a problem there during the resize event and the scroller.resize();
Any advice on how to implement a good resize function with scrollama to avoid that issue ?
I'm trying to understand on how to deal with it for the last couple of days and it's driving me mad. Everything is working fine, except for those double active steps ...
js:
//DEBOUNCE
function debounce(func, wait, immediate) {
	var timeout;
	return function() {
		var context = this, args = arguments;
		var later = function() {
			timeout = null;
			if (!immediate) func.apply(context, args);
		};
		var callNow = immediate && !timeout;
		clearTimeout(timeout);
		timeout = setTimeout(later, wait);
		if (callNow) func.apply(context, args);
	};
};

var container = document.querySelector('#scroll');
var text = container.querySelector('.scroll__text');
var steps = text.querySelectorAll('.step');

// load
var scroller = scrollama();

// call on resize
function handleResize(){
  scroller.resize();
}
// Resize event
// --> y or n
function size() {
    if(window.matchMedia("(min-width:769px)").matches) {
        handleResize();
        scroller.enable();
    } else {
        handleResize();
        scroller.disable();
    };
  };

// --> call on resize
var debounce_resize = debounce(function() {
  size();
}, 250);

// event enter
function handleStepEnter(response) {
  response.element.classList.add('is-active');
  console.log('is-enter');
}

// event exit
function handleStepExit(response) {
  response.element.classList.remove('is-active');
  console.log('is-out');
}

// init
// --> call on DOMloaded
function init() {

  // -->
  size();

  scroller.setup({
    step: '.scroll__text .step',
    debug: true,
  })
    .onStepEnter(handleStepEnter)
    .onStepExit(handleStepExit);

  // setup resize event
  window.addEventListener('resize', debounce_resize);
}

// GO
document.addEventListener("DOMContentLoaded", function(event) {
    init();
  });

Let me know,
Many thanks !

I just tested out resizing and it seems to be working as intended. I'm in latest Chrome. What browser/version are you on?


LoadError: Failed to precompile JuMPeR

using JuMPeR
INFO: Precompiling module JuMPeR...
WARNING: Method definition /(JuMP.GenericAffExpr{Float64, JuMPeR.Uncertain}, JuMP.GenericNormExpr{#P<:Any, #C<:Any, #V<:JuMPeR.Uncertain}) in module JuMPeR at /home/juser/.julia/v0.5/JuMPeR/src/robustops.jl:173 overwritten at /home/juser/.julia/v0.5/JuMPeR/src/robustops.jl:183.
WARNING: Method definition /(JuMP.GenericAffExpr{JuMP.GenericAffExpr{Float64, JuMPeR.Uncertain}, JuMP.Variable}, JuMP.GenericNormExpr{#P<:Any, #C<:Any, #V<:JuMPeR.Uncertain}) in module JuMPeR at /home/juser/.julia/v0.5/JuMPeR/src/robustops.jl:206 overwritten at /home/juser/.julia/v0.5/JuMPeR/src/robustops.jl:219.
ERROR: LoadError: LoadError: in @LinearConstraint (new_lhs <= unc_con.ub): expected comparison operator (<=, >=, or ==).
in include(::ASCIIString) at ./boot.jl:240
in include_from_node1(::ASCIIString) at ./loading.jl:417
in include(::ASCIIString) at ./boot.jl:240
in include_from_node1(::ASCIIString) at ./loading.jl:417
[inlined code] from ./boot.jl:243
in anonymous at ./:4294967295
in eval(::Module, ::Any) at ./boot.jl:243
[inlined code] from ./sysimg.jl:11
in process_options(::Base.JLOptions) at ./client.jl:239
in _start() at ./client.jl:318
while loading /home/juser/.julia/v0.5/JuMPeR/src/oracle.jl, in expression starting on line 72
while loading /home/juser/.julia/v0.5/JuMPeR/src/JuMPeR.jl, in expression starting on line 262
LoadError: Failed to precompile JuMPeR to /home/juser/.julia/lib/v0.5/JuMPeR.ji
while loading In[2], in expression starting on line 5
in error(::ASCIIString) at ./error.jl:21
in compilecache(::ASCIIString) at ./loading.jl:496
in compilecache(::Symbol) at ./loading.jl:485
in require(::Symbol) at ./loading.jl:355
[inlined code] from ./essentials.jl:82
in include_string(::ASCIIString, ::ASCIIString) at ./loading.jl:371

Cc @IainNZ

This looks like 0.5, which JuMPeR doesn't support (and maybe not any release version of JuMP either?). I'm not planning on support 0.5 until a prerelease comes out.

@mandarup you should install Julia locally and use 0.4 - all the software will work just fine.

Ah I see Julia 0.4.5 is supported on JuliaBox, but I got some cryptic errors
INFO: Recompiling stale cache file /home/juser/.julia/lib/v0.4/Calculus.ji for module Calculus.
WARNING: Module Compat uuid did not match cache file
  This is likely because module Compat does not support  precompilation but is imported by a module that does.
WARNING: deserialization checks failed while attempting to load cache from /home/juser/.julia/lib/v0.4/Calculus.ji
INFO: Precompiling module Calculus...
INFO: Recompiling stale cache file /home/juser/.julia/lib/v0.4/Calculus.ji for module Calculus.
WARNING: Module Compat uuid did not match cache file
  This is likely because module Compat does not support  precompilation but is imported by a module that does.

Which I have no idea about, so again, definitely recommend local install.

@IainNZ did you get this error in IJulia? If yes, it is probably because IJulia already loads Compat. Using the console to update packages would avoid that.

Yes IJulia.
I'm not sure how anyone would know to do that - maybe worth putting some sort of notice on the page about installing packages?

Yes. I had added it (https://github.com/JuliaLang/JuliaBox/commit/c01276721abe3273e21c373467052b25f350024b) to the place where we list the installed packages on the JuliaBox settings page. But that's probably not the best place for that...
Please do mention if you think of something else more appropriate.


Spec and Implement Release Metadata Format

The release metadata is the information that annotates a differentially private report, and includes things like epsilon, delta, report value, error bounds.
We need a spec describing the fields and semantics, and optionally serialization.  The semantics should be shared across different modules as much as possible, and serialization at a minimum will be important for implementing a report store.
Yarrow format is: https://github.com/privacytoolsproject/yarrow/blob/master/prototypes/release.proto
Example of getting release metadata in Yarrow:
https://github.com/privacytoolsproject/PSI-Library/blob/develop/vignettes/dp-mean.Rmd

Initial draft spec at https://docs.google.com/document/d/1PTAG2xImB5B3m4crc9t3MQLyRUp3GieD-u8tJlNCDzY/edit#heading=h.ga5nyepy7ehj


Spreadsheet with no hyperlinks creates xlsx with "unreadable content"

When you create a workbook with no hyperlinks, the worksheet is created with an empty < hyperlinks > tag which Excel considers invalid.

Thanks, merged #5.