InvalidArgumentError : Expected image (JPEG, PNG, or GIF), got unknown format starting with '255'

System information

OS Platform and Distribution  :Linux Ubuntu 16.04
TensorFlow installed from  :binary
TensorFlow version  :1.1.0
Python version : 2.7.12

Describe the problem
trying to feed a model an image encoded in string as the model require that as an input string
signature_def {
  key: "serving_default"
  value {
    inputs {
      key: "image_bytes"
      value {
        name: "Placeholder:0"
        dtype: DT_STRING
        tensor_shape {
          dim {
            size: -1
          }
        }
      }
    }
    inputs {
      key: "key"
      value {
        name: "Placeholder_1:0"
        dtype: DT_STRING
        tensor_shape {
          dim {
            size: -1
          }
        }
      }
    }
    outputs {
      key: "key"
      value {
        name: "Identity:0"
        dtype: DT_STRING
        tensor_shape {
          dim {
            size: -1
          }
        }
      }
    }
    outputs {
      key: "prediction"
      value {
        name: "ArgMax:0"
        dtype: DT_INT64
        tensor_shape {
          dim {
            size: -1
          }
        }
      }
    }
    outputs {
      key: "scores"
      value {
        name: "final_ops/softmax:0"
        dtype: DT_FLOAT
        tensor_shape {
          dim {
            size: -1
          }
          dim {
            size: 3
          }
        }
      }
    }
    method_name: "tensorflow/serving/predict"
  }
}

Here is the code :-
def load_image( infilename ) :
    img = Image.open( infilename )
    img.load()
    data = np.asarray( img, dtype="string" )
    return data

export_dir = '.'
with tf.Session(graph=tf.Graph()) as sess:
	model = tf.saved_model.loader.load(sess, ['serve'], export_dir)
	input_dict, output_dict =_signature_def_to_tensors(model.signature_def['serving_default'])
	out = sess.run(output_dict, feed_dict={input_dict['image_bytes']: load_image("fullsize.jpeg").flatten()})
	print(input_dict)

Error returns  is :
InvalidArgumentError (see above for traceback): Expected image (JPEG, PNG, or GIF), got unknown format starting with '255'
	 [[Node: map/while/DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](map/while/TensorArrayReadV3)]]

how i can solve that , any help

The error message suggests that the input isn't a valid JPEG image.
Are you sure that the image you're feeding (which seems to be a 3 byte file) is a valid JPEG image?

I feed the Image using np.array and also tried to feed it with tf.image.decode_jpeg getting an error in both cases . when i am using tf.image.decode_jpeg as
image_contents = tf.read_file("fullsize.jpeg")
image = tf.image.decode_jpeg(image_contents, channels=3)

i get this error
    raise TypeError('The value of a feed cannot be a tf.Tensor object. '
TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, numpy ndarrays, or TensorHandles.

how to validate if the image is valid or not and also the best way to feed the image .
Thanks In Advance

Sorry , Edited the Error Message i tried it again with the same values but represent this error , seems to be problem in the encoding

This question is better asked on  StackOverflow since it is not a  bug or feature request. There is also a larger community that reads questions there. Thanks!
That said, a few things to note:

tf.image.decode_jpeg expects the encoded-JPEG bytes (i.e., the contents of the file). The use of PIL and in particular PIL.Image.load() suggests that the numpy array you're creating consists of the decoded contents (decoded by PIL). Which is why the sess.run() call is failing as it expects an encoded JPEG but you're providing it a decoded one.

To provide the raw contents of the file as a feed, you'd want to do something like so:

def load_image(filename):
  with open('fullsize.jpeg') as f:
    return np.array(f.read())

Which will create the numpy array of the encoded JPEG bytes that you can feed to your session.
Hope that helps.

@engahmed1190 ,  I meet a silimar question with you when i train models on google TensorFlow Object Detection API, May i ask if the question has been solved? what's the problem and how to solved? thank you!

@keyankarthisdk , maybe the reason is some picture is not the JPEG format. You can use tf.image.decode_jpeg() to test which picture is not the right format for tensorflow and then discard. I use this way solve the problem.

Thank you so much
On Mon, 11 Feb 2019, 3:22 pm clovking<EMAIL_ADDRESS>wrote:

@keyankarthisdk https://github.com/keyankarthisdk , maybe the reason is
some picture is not the JPEG format. You can use tf.image.decode_jpeg() to
test which picture is not the right format for tensorflow and then discard.
I use this way solve the problem.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/13583#issuecomment-462267255,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AoFJhC4L3sn7W6llk8rtJD----IaO94Dks5vMT11gaJpZM4PyhJ8
.

``Hello, I've been running through a similar problem and

@keyankarthisdk , maybe the reason is some picture is not the JPEG format. You can use tf.image.decode_jpeg() to test which picture is not the right format for tensorflow and then discard. I use this way solve the problem.

Like where should i use the decode_jpeg, because I'am working with Application API
Here is my code,
base_model=tf.keras.applications.EfficientNetB2(include_top=False, input_shape=(224, 224, 3))
base_model.trainable=False
inputs=tf.keras.layers.Input(shape=(224, 224, 3))
x=tf.keras.layers.GlobalAveragePooling2D()(inputs)
outputs=tf.keras.layers.Dense(2, activation='softmax')(x)
model_0=tf.keras.Model(inputs, outputs)
model_0.compile(loss=tf.keras.losses.binary_crossentropy,
                optimizer=tf.keras.optimizers.Adam(),
                metrics=['accuracy'])
model_0.fit(data_gen, epochs=5)

And when i executed this code I was getting this sort of error,

InvalidArgumentError                      Traceback (most recent call last)
 in ()
2                 optimizer=tf.keras.optimizers.Adam(),
3                 metrics=['accuracy'])
----> 4 model_0.fit(data_gen, epochs=5)

6 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58     ctx.ensure_initialized()
59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
61   except core._NotOkStatusException as e:
62     if name is not None:
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument:  Unknown image file format. One of JPEG, PNG, GIF, BMP required.
[[{{node decode_image/DecodeImage}}]]
[[IteratorGetNext]]
[[IteratorGetNext/_4]]
(1) Invalid argument:  Unknown image file format. One of JPEG, PNG, GIF, BMP required.
[[{{node decode_image/DecodeImage}}]]
[[IteratorGetNext]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_216368]
Function call stack:
train_function -> train_function
Thanks in advance

@keyankarthisdk , maybe the reason is some picture is not the JPEG format. You can use tf.image.decode_jpeg() to test which picture is not the right format for tensorflow and then discard. I use this way solve the problem.

Hello,
I'am currently working with Application API and where should i Insert the decode_jpeg function my code is as follows
base_model=tf.keras.applications.EfficientNetB2(include_top=False, input_shape=(224, 224, 3))
base_model.trainable=False
inputs=tf.keras.layers.Input(shape=(224, 224, 3))
x=tf.keras.layers.GlobalAveragePooling2D()(inputs)
outputs=tf.keras.layers.Dense(2, activation='softmax')(x)
model_0=tf.keras.Model(inputs, outputs)

model_0.compile(loss=tf.keras.losses.binary_crossentropy,
                optimizer=tf.keras.optimizers.Adam(),
                metrics=['accuracy'])
model_0.fit(data_gen, epochs=5)

And when I executed this code I was getting this error
InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-101-6769bac1fcee> in <module>()
      2                 optimizer=tf.keras.optimizers.Adam(),
      3                 metrics=['accuracy'])
----> 4 model_0.fit(data_gen, epochs=5)

6 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     58     ctx.ensure_initialized()
     59     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60                                         inputs, attrs, num_outputs)
     61   except core._NotOkStatusException as e:
     62     if name is not None:

InvalidArgumentError: 2 root error(s) found.
  (0) Invalid argument:  Unknown image file format. One of JPEG, PNG, GIF, BMP required.
	 [[{{node decode_image/DecodeImage}}]]
	 [[IteratorGetNext]]
	 [[IteratorGetNext/_4]]
  (1) Invalid argument:  Unknown image file format. One of JPEG, PNG, GIF, BMP required.
	 [[{{node decode_image/DecodeImage}}]]
	 [[IteratorGetNext]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_216368]

Function call stack:
train_function -> train_function

Please help me with this
Thanks in advance

@rohitkumar9989  you can solve this problem with the below code:
import cv2
import tensorflow as tf
import os

def check_images(s_dir, ext_list):
    bad_images=[]
    bad_ext=[]
    s_list= os.listdir(s_dir)
    for klass in s_list:
        klass_path=os.path.join (s_dir, klass)
        print ('processing class directory ', klass)
        if os.path.isdir(klass_path):
            file_list=os.listdir(klass_path)
            for f in file_list:
                f_path=os.path.join (klass_path,f)
                index=f.rfind('.')
                ext=f[index+1:].lower()
                if ext not in ext_list:
                    print('file ', f_path, ' has an invalid extension ', ext)
                    bad_ext.append(f_path)
                if os.path.isfile(f_path):
                    try:
                        img=cv2.imread(f_path)
                        shape=img.shape
                        image_contents = tf.io.read_file(f_path)
                        image = tf.image.decode_jpeg(image_contents, channels=3)
                    except Exception as e:
                        print('file ', f_path, ' is not a valid image file')
                        print(e)
                        bad_images.append(f_path)
                else:
                    print('*** fatal error, you a sub directory ', f, ' in class directory ', klass)
        else:
            print ('*** WARNING*** you have files in ', s_dir, ' it should only contain sub directories')
    return bad_images, bad_ext

For execution of the above code:
source_dir =r'/data/validation/'
good_exts=['jpg','jpeg'] # list of acceptable extensions
bad_file_list, bad_ext_list=check_images(source_dir, good_exts)
if len(bad_file_list) !=0:
    print('improper image files are listed below')

print(bad_file_list)
print(bad_ext_list)

Yes thank you sir....  I've actually made a helper function that converts
all the images into JPEG format... So I wasn't able to see any error....
On Thu, 10 Jun 2021, 16:01 Deep Modi, @.***> wrote:

@rohitkumar9989 https://github.com/rohitkumar9989 you can solve this
problem with the below code:
import cv2
import tensorflow as tf
import os
def check_images(s_dir, ext_list):
bad_images=[]
bad_ext=[]
s_list= os.listdir(s_dir)
for klass in s_list:
klass_path=os.path.join (s_dir, klass)
print ('processing class directory ', klass)
if os.path.isdir(klass_path):
file_list=os.listdir(klass_path)
for f in file_list:
f_path=os.path.join (klass_path,f)
index=f.rfind('.')
ext=f[index+1:].lower()
if ext not in ext_list:
print('file ', f_path, ' has an invalid extension ', ext)
bad_ext.append(f_path)
if os.path.isfile(f_path):
try:
img=cv2.imread(f_path)
shape=img.shape
image_contents = tf.io.read_file(f_path)
image = tf.image.decode_jpeg(image_contents, channels=3)
except Exception as e:
print('file ', f_path, ' is not a valid image file')
print(e)
bad_images.append(f_path)
else:
print('*** fatal error, you a sub directory ', f, ' in class directory ', klass)
else:
print ('*** WARNING*** you have files in ', s_dir, ' it should only contain sub directories')
return bad_images, bad_ext
For execution of the above code:
source_dir =r'/data/validation/'
good_exts=['jpg','jpeg'] # list of acceptable extensions
bad_file_list, bad_ext_list=check_images(source_dir, good_exts)
if len(bad_file_list) !=0:
print('improper image files are listed below')
print(bad_file_list)
print(bad_ext_list)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/13583#issuecomment-858508478,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQS5W25IOPN4KRIJ4CF2PO3TSCIBNANCNFSM4D6KCJ6A
.

@rohitkumar9989 Okay, That's great. :)

Awesome, thanks!
Thanks for your handy help...
On Thu, 10 Jun 2021, 16:07 Deep Modi, @.***> wrote:

@rohitkumar9989 https://github.com/rohitkumar9989 Okay, That's great. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/13583#issuecomment-858511996,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQS5W23LQD2EAC3XBJHOGQTTSCIX3ANCNFSM4D6KCJ6A
.

@rohitkumar9989  could you please tell me how did you build the helper class?
Does this mean if I only have .jpg files to train on,I should be able to do?

I got a similar error message.

The problem was that I had several corrupt files that the program couldn't handle. I deleted the files and it works

I got a similar error message.

The problem was that I had several corrupt files that the program couldn't handle. I deleted the files and it works


Broken stacktrace

After recent changes, stacktrace is not correctly formed. Example of stacktrace:
Uncaught <error: TypeError: Cannot read properties of null (reading 'toString')>
    at $rt_exception (http://localhost:9090/js/classes.js:256:31)
    at $rt_throw (http://localhost:9090/js/classes.js:245:11)
    at $rt_nullCheck (http://localhost:9090/js/classes.js:895:9)

It fails no matter what exception was thrown from Java code in this code snippet
function $rt_exception(ex) {
    var err = ex.$jsException;
    if (!err) {
        var javaCause = $rt_throwableCause(ex);
        var jsCause = javaCause !== null ? javaCause.$jsException : $rt_globals.undefined;
        var cause = typeof jsCause === "object" ? { cause : jsCause } : $rt_globals.undefined;
        err = new JavaError("Java exception thrown", cause);
        if (typeof $rt_globals.Error.captureStackTrace === "function") {
            $rt_globals.Error.captureStackTrace(err);
        }
        err[$rt_javaExceptionProp] = ex;
        ex.$jsException = err;
        $rt_fillStack(err, ex);
    }
    return err;
}

in the line
$rt_globals.Error.captureStackTrace(err);

and source maps are not loaded in this case.

@Ihromant, could not reproduce. Can you please provide a self-contained example?

https://github.com/Ihromant/hexagon/tree/test5
Only what I'm doing - throwing exception. Running project in newest IDEA, teavm-0.9.0-dev-3, xubuntu, Chromium browser in incognito mode.
Uncaught <error: TypeError: Cannot read properties of null (reading 'toString')>
    at $rt_exception (http://localhost:9090/teavm/classes.js:256:31)
    at $rt_throw (http://localhost:9090/teavm/classes.js:245:11)
    at ui_Client_main (http://localhost:9090/teavm/classes.js:1690:5)
    at http://localhost:9090/teavm/classes.js:598:15
    at $rt_startThread (http://localhost:9090/teavm/classes.js:1891:18)
    at http://localhost:9090/teavm/classes.js:597:9
    at window.$teavm_deobfuscator_callback [as run] (http://localhost:9090/teavm/classes.js:2045:29)
    at Hz (http://localhost:9090/teavm/classes.js.deobfuscator.js:193:412)
    at EC.<anonymous> (http://localhost:9090/teavm/classes.js.deobfuscator.js:30:397)
    at XMLHttpRequest.fn (http://localhost:9090/teavm/classes.js.deobfuscator.js:89:87)

Full stacktrace.

@konsoletyper


cache prefix

Is your feature request related to a problem? Please describe.
Currently im using redis server with multiple project, when i need to clean all cache related project A for eg, it its quite hard because we can't identify which cache belong to.
Describe the solution you'd like
it will be good if we have the cache prefix that is configurable via module for example we can set to nestjs:* so when we need to delete the cache, we can delete it by prefix and the other cache that is not related to a specific project is not deleted

@tarikhagustia I do believe that as we integrate Keyv (https://keyv.org) into this project this feature will happen. Our goal is to implement it over the next 3-4 months. FYI


[COMMUNITY] junrushao1994 -> committer

Please join us to welcome Junru Shao(@junrushao1994) as a new Committer. Junru has been actively contributing to various aspects of the TVM codebase. He reimplemented and refactored the Target system which greatly helped code lowering and code generation. Junru also largely contributed to the runtime and node objects to improve the object passing. From the frontend side, Junru has also implemented some Relay operators.
In addition, Junru has been very actively sharing his thoughts in the discussion forum to help improve the design of new features and provide new developers with better understanding of TVM. He has been constantly providing suggestive comments to the pull request.

Commits History
Code Review
Community Forum Summary

Thank you SO much for offering such a great opportunity!! I am super thrilled to serve the community as an Apache TVM committer! It is always great experience chatting and sharing ideas in the friendly TVM forum, and let's continue to work together to make TVM more accessible, easy-to-use as well as to keep our technique the SOTA in deep learning compilation!

Congrats @junrushao1994!!!

@zhiics you will probably have to rebase due to auto scheduling CI corruption


IPFS-request

Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.Screenshot of IPFS command-line interface
Command-line install
All IPFS, no frills
Just want to use IPFS from your terminal? Follow these step-by-step instructions for getting up and running on the command line using the Go implementation of IPFS.

👀


Add Simple Intrest Calculator

Is your feature request related to a problem? Please describe.
Adding A Simple Intrest Calculator
Describe the solution you'd like
Adding A Simple Intrest Calculator
Describe alternatives you've considered
Adding A Simple Intrest Calculator

I want to Work On the issue

@Sulagna-Dutta-Roy Please Assign me the issue


Style new docs header

Hello! Here is the new proposed style for the Docs header.
In the sample image we can see icons to the right (we will set a placeholder for up to 4 icons) even though we will not put any icons right now.

The idea is to modify the following files to achieve this: Docfiles/docs.html, Docsfiles/header.html and theme/themes/docs.less

what will it look like in mobile?
what about the current table of contents?

For mobile we are still designing a proposal mainly because of the space of the "breadcrumb" control.

For the TOC this is how it will look like.

(Notice the filter box is actually the current search box).
Let me prepare a new set of clean design screenshots and I'll add those here. :)

Hello! Here is an update on the Docs Design.
This one is the desktop version:

We are proposing a main header that unifies the experience in Docs and in Editor
Since the Search box gives results globally and not only about docs we are currently proposing placing the box in the top right.
Even though we are not placing a filter box for the TOC content we might consider it later.
The Print and Language buttons are placed in what we call an action "area" or right "row" where we could place other content actions like "share"
The logo and accent color still be defined in "appTheme" configuration. We can discuss later if we would need other configuration options, like a color palette or "target" specific styles.

Now, the next set of images are the proposed mobile "responsive" experience.

The breadcrumb control is scrollable when the width doesn't fit the device.

The hamburger menu displays the TOC.

The search box "lives" in the hamburger menu.

When we choose language we have a selection dialog appearing.

These last pdfs have other Explorations. Please, feel free to comment and propose other explorations.
Makecode - Documentation  (Mobile) - Breadcrumb.pdf
Makecode - Documentation  (Mobile).pdf

While we were exploring the language picker we found that's rendered by react lang.tsx. What do you think about recreating this with vanillajs/jquery/semantic-ui in a new file docfiles/controls/langPicker.js
The idea is to use appTheme.selectLanguage and appTheme.availableLocales to get the same options as in the editor. There is a const 'allLanguages' in lang.tsx we could replicate that one or figure a way to refactor that object to be able to get it from docs. Also for the look and feel the idea is to make both pickers look the same.


Backup settings page

The administrator(s) should be able to manage the backup settings true a web form

@ghermans Can u make the view?

Sure


typo in preface?

Page xxviii: "To make this introductory book more self-containedv and useful for students,"  -> To make this introductory book more self-contained and useful for students,"

Should've checked closed issues


[PE-D] help command is incomplete

The help command does not show the command list as described in the UG.

Labels: severity.Medium type.FunctionalityBug
original: honganhcs/ped#10

dupe of #165


Reject spans that create a circular loop

Description
Spans have a parent. A span is not allowed reference itself or one of its anscetors as a parent.
Data sent to the dashboard could contain an invalid circular reference. Rendering the bad data in the UI could cause errors. This PR adds validation logic when spans are received to reject the bad data.
Checklist

Is this feature complete?

[x] Yes. Ready to ship.
[ ] No. Follow-up changes expected.

Are you including unit tests for the changes and scenario tests if relevant?

[x] Yes
[ ] No

Did you add public API?

[ ] Yes

If yes, did you have an API Review for it?

[ ] Yes
[ ] No

Did you add <remarks /> and <code /> elements on your triple slash comments?

[ ] Yes
[ ] No

[x] No

Does the change make any security assumptions or guarantees?

[ ] Yes

If yes, have you done a threat model and had a security review?

[ ] Yes
[ ] No

[x] No

Does the change require an update in our Aspire docs?

[ ] Yes

Link to aspire-docs issue:

[x] No

Microsoft Reviewers: Open in CodeFlow

PR feedback addressed.


Add new support for Seeeduino Cortex-M0+

This fulfills issue #148 for Bridle main line together with some improvements in the "helloshell" example.

new board documentation for preview: Seeeduino Cortex-M0+
new board support for USB console: Board specific identifiers
new board support for Grove connector stack: Build and Programming –> Atmel Semiconductors –> Seeeduino Cortex-M0+

LED Blinky
LED Fade
LED Button


deny warnings only when compiling for tests

deny warnings only when compiling for tests
This makes cargo forward-compatible with lint changes upstream in rust.
This relates to rust-lang/rust/pull/33091. This PR makes cargo never stop compiling from denied lints, but tests may still fail. This allows using cap-lints warn sensibly in rust's cargotest, since we know it will not break cargo itself.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @huonw (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

cc @alexcrichton

So this is the simplest approach and needs upstream rust to use cap-lints.
Another approach could be to never use deny (neither build nor test), but opt into it when running CI.

Would this actually help the compiler? We always run cargo test anyway which would deny warnings eventually?

This allows using cap-lints warn sensibly in rust's cargotest, since we know it will not break cargo itself.

I'm not sure I understand, if we use --cap-lints warn then it essentially ignores these #[deny] directives, so why would we also need to change them?

Please see the issue rust-lang/rust/pull/33091 for context and also the wish expressed here https://github.com/rust-lang/rust/pull/33091#issuecomment-214845797
With no deny in the regular build, lint changes will not break cargo. Thus we can accept using cap-lints warn in cargotest like @brson wishes. I think it makes sense.
Alternatively, we can change cargo around to not use deny at all in the default build / test configuration and only enable it in cargo's CI. I thought it would be unbeautiful to add some configuration variable in cargo code just for that.

Ok, sorted this out on IRC with @bluss.
Specifically:

The combination of https://github.com/rust-lang/cargo/pull/2617 and https://github.com/rust-lang/rust/pull/33215 means that no lint changes will bounce Rust code.
This PR should guarantee that cargo build then always works, but not necessarily cargo test.

I think that basically only guaranteeing cargo build probably isn't too useful for now, as we still wouldn't be able to land changes until cargo test was fixed. As a result I'm gonna close this for now, but we can certainly revisit if it becomes a problem

I did mention the alternative above

Another approach could be to never use deny by default (neither build nor test), but opt into it when running CI.

from our discussion I think it's the best approach.

In that sentence, CI is this repository's CI.

Unfortunately that still wouldn't help landing changes in Cargo (as they'd still be blocked on broken lints) :(

improved lints :wink:

right!


Write more tests

[ ] lib/callbacks
[ ] lib/data
[ ] lib/fp
[ ] lib/id
[ ] lib/math
[ ] lib/network
[ ] lib/oop
[ ] lib/sort
[ ] lib/strings
[ ] lib/time
[ ] lib/units
[ ] lib/utilities

Is this task actual? For example, I don't see tests for
lib/fp lib/oop
and what is about other, should I add some tests or make them more exact?


lavavu crashes kernel on jupyterhub

Not sure why ...

v = 1.6.7

And 1.8+

Seems to be sporadic ... keep trying to launch (maybe pause after xvfb starts)


MatchError on channel messages

Hi,
I'm facing this kind of error for a while now, and I was wondering what could be its cause.
[chatty] :Uniaika!~xxx@xxxxxxx PRIVMSG #FoobarLol :foobarlol
[chatty] %MatchError{term: "foobarlol"}

There is a single place where errors are caught and logged, here.
The error comes either from HookHelpers or from a custom hook you have installed. You can try removing the try block to get a trace when the error happens next time.

I shall do that ;) thank you!

@Annwenn did you find out what the error was?

@alco I'd bet on an implementation error  from Queertoo Dev Team's side. And it's not relevant anymore with the merge of otp.


UseTriggers in startUp

Hi
i have million rows in Cities, States, Countries. However when i user trigger in start this slows how my initial seeding setup. I have waited like 10 minutes and then i have to comment below lines and then uncomment after initial seeding. I'm not sure whats happening under the hood that makes seeding so slow. I am not even triggering any event during Cities, States and Countries seeding.
// .UseTriggers(triggerOptions =>
            // {
            //     triggerOptions.AddTrigger<DeleteCustomerDocumentTrigger>();
            //     triggerOptions.AddTrigger<DeletePhysicialFileTrigger>();
            //     triggerOptions.AddTrigger<DeleteProviderDocumentTrigger>();
            // })

Hi Hassan, I presume that you don't actually seed all millions of rows in during application startup? If so, you may be better off using something like BulkInsert from https://github.com/borisdj/EFCore.BulkExtensions
Adding 10 minutes however is not to be expected. Having triggers causes a slight overhead to your DbContext but unless you're actually invoking triggers, this should overhead should hardly be noticeable.
Some suggestions as to what you may try:

Make sure that you're not accidentally invoking Triggers. You can still make a call to UseTriggers without having any trigger registered. It should then cause no noticeable performance overhead. Alternatively you may observe the Info level logs produced under scope EntityFrameworkCore.Triggered to see what is happening internally.
Make sure that you don't have Info level logging enabled for Triggers when inserting large volumes of data.
Implement Trigger interface explicitly (e.g. IBeforeSaveTrigger) instead of using the Trigger base class from the Extension package. The latter will cause some additional performance overhead.

Let me know if the above helps.


Fixed issues around the internal thread and cleanup

Found some issues when disconnecting a remote endpoint...

Current coverage is 100% (diff: 100%)

Merging #1 into master will not change coverage

@@           master    #1   diff @@
===================================
  Files           4     4
  Lines          59    61     +2
  Methods         0     0
  Messages        0     0
  Branches        0     0
===================================
+ Hits           59    61     +2
  Misses          0     0
  Partials        0     0

Powered by Codecov. Last update 5b70ee7...aab87c7


[Back-Port Request - 1.19.3] For Mod: Mob-Death-Sounds

Hey, @Merphyx, could you back-port the mod (name in title), to 1.19.3? Why 1.19.3? Because that is my main client, and I really seem to like your mod. Since 1.19.3 & 1.20.4 are very different, could you make the mod ONLY play a sound (minecraft:entity.experience_orb.pickup), at a default pitch (1), when a mob kill stat is increased? So like no commands or anything, so it could be easier for you, if you are ok with this request?

Done.
https://github.com/Merphyx/Mob-Death-Sounds-Fabric/releases/download/V1.0-1.19.3/mob-ds-v1.0+mc1.19.3.jar

@Merphyx, thank you so much! When I come back home, I'll download & run the mod, and let you know if any issues/crashes come up.

Hey, @Merphyx, I just came home, ran the mod, and I gotta say - the mod works great, no crashes or whatnots! Thank you so much for your time and effort.


async_close do not recv peer close frame

I use async_close to close the socket, but after 5min recv callback, have any method to force close the connection, and not wait the close frame
may be I should call
std::optional<boost::beast::websocket::stream<
    boost::beast::ssl_stream<boost::beast::tcp_stream>>> ws_stream_;
//force close?
boost::beast::get_lowest_layer(*ws_stream_).cancel();

after that I want to do reconnect
ws_stream_.emplace(exec_, ctx_);

it is right?
thanks

Have you set a timeout on the stream? https://www.boost.org/doc/libs/1_85_0/libs/beast/doc/html/beast/using_websocket/timeouts.html

Have you set a timeout on the stream? https://www.boost.org/doc/libs/1_85_0/libs/beast/doc/html/beast/using_websocket/timeouts.html

I do not use timeout, I want to close it quickly and reconnect,  just like force close not graceful, is there some method?

beast::get_lowest_layer(*ws_stream_).cancel(); will do it, or you can simply ignore the stream object and let it be destroyed.

beast::get_lowest_layer(*ws_stream_).close(); will do it, or you can simply ignore the stream object and let it be destroyed.

you mean direct use ws_stream_.emplace(exec_, ctx_);  let it destroy?

Yes, that would have the same effect as closing the underlying socket.


Chore: Add tests for rpcproxy

Description
Currently I'm just deploying it into qa and seeing if things work as I would expect. Unit testing grpc stuff isn't fun, but we'll want to add those in once we confirm things are mostly working.
Most of the endpoints just proxy the requests back and forth, but there's additional logic in initialize_session that definitely warrants tests.

We can do this in next milestone


Remove broken docs links

:tada: This issue has been resolved in version 0.79.2 :tada:
The release is available on:

v0.79.2
GitHub release

Your semantic-release bot :package::rocket:


Hide hidden folders like ".git" in the navigation

I have a git repo only for the content folder, which leads to the content folder having a ".git" folder which is shown in the backend navigation. I think in general it would be good to hide such folders which names start with a ".".

thank you!


includeLayer options for Leaflet DVF Chloropleth not working for numeric data ranges

I am having problem with implementing the Leaflet DVF includeLayer function (Equivalent to filter) as follows for a Chloropleth GeoJSON Data Layer:
includeLayer: function(feature) {
return (feature.Total_Population <= 1000) && (feature.Total_Population >= 500);
}
Unfortunately there isn't much documentation online. From what i've read, it seems that this can only be applied on string type data and not on numbers. Is this so? Or am I missing something? Thanks in advance for any assistance regarding this issue!

That should definitely work.  Is feature in this case a GeoJSON feature?  If that's true, then it should probably be:
includeLayer: function(feature) {
     return (feature.properties.Total_Population <= 1000) && (feature.properties.Total_Population >= 500);
}

What happens if you open the Chrome or Firefox dev tools and put a breakpoint on the return line?  What does feature.Total_Population look like?

Darn missed the properties part ... It fixed up my coding, thank you so much Scott!


 How should I do it ?

const auto response = request.send("GET", body, { { "Authorization", "value"}})

Since the headers list is a map of string,string you just need to wrap each pair in the parent set of braces.
const auto response = request.send("GET", body, { { "Authorization", "value" } });


docs(blog): Maker announcement

PR Checklist
Please review the guidelines for contributing to this repository.

[X] I am requesting to pull a topic/feature/bugfix branch (right side). In other words, not master.
[X] I have run npm test against my changes and tests pass.
[X] I have added or edited necessary documentation, or no docs changes are needed.

Description
Maker announcement post

Looks good pending E's review.

@roppem9 do you want to add a hero image now before merging, or is it still in the works?

I can work on one this afternoon and add it

@noelweichbrodt Just added a hero img


ensure that tropical ANPP data from Taylor et al. 2017 is included

Taylor et al. 2017 compiled tropical ANPP estimates, and data are published here. This effort was independent of ours. There's probably a fair amount of overlap, but it looks like they have some records we don't.

also, this publication (probably everything in here is in Taylor):
Cleveland, C. C., A. R. Townsend, P. Taylor, S. Alvarez-Clare, M. M. Bustamante, G. Chuyong, S. Z. Dobrowski, P. Grierson, K. E. Harms, B. Z. Houlton, A. Marklein, W. Parton, S. Porder, S. C. Reed, C. A. Sierra, W. L. Silver, E. V. Tanner, and W. R. Wieder. 2011. Relationships among net primary productivity, nutrients and climate in tropical rain forest: a pan-tropical analysis. Ecology Letters 14:939-947.

@beckybanbury, this is one of the data sources I mentioned yesterday. I just wrote Cory Cleveland about these data; awaiting response.

@teixeirak, great, let me know when you hear back! Do we have the FLUXNET data saved anywhere; so far I've just found the site list? If not, I will download.

From Cory Cleveland:
The data are public; we're welcome to use them. There's an updated database with much more data in the works, and he's checking with collaborators as to whether we can use that.
This link should work to the old database. There are four excel files (litterfall [“canopy” column]; stem growth [“woody” column; decomposition [“k” column]; and ANPP [“ANPP” column] in the respective files). The metadata file at this link is still broken but the labels should be self-explanatory (MAP, MAT, Lat, Lon, Reference, Country, etc.). The ID # is only relevant to our specific analysis. He's happy to field questions.

@teixeirak, there is some data in this set from a dissertation that has not been published in a journal as far as I can tell; do we include this or not?

Yes, data that are published in intermediary sources are counted as meeting the “published criteria, even if personal communication.

Great! What should I enter in the citation.ID column for these sources?

If there’s a full citation available, create an entry in citations. If personal comm, put the published source (here, Taylor) and list personal comm source in notes.

@teixeirak, I have now completed entering the litterfall, stem growth and total ANPP data from this database.

Wonderful, thank you!

Wonderful, thank you!


The 'Masa.BuildingBlocks.Dispatcher.Events.IMiddleware' class name and 'Microsoft.AspNetCore.Http.IMiddleware' name repetition

At first glance, I thought it was Microsoft middleware. In fact, this middleware of MASA is the command middleware of CRQS and other purposes. Could you please make a distinction so as to facilitate understanding of the concept

@zhenlei520 what do you think?

Are you think 'IMasaEventMiddleware' this name is ok? or can you give it better name?

Are you think 'IMasaEventMiddleware' this name is ok? or can you give it better name?

You can use IEventMiddleware, if it is convenient for you, can you contribute pr

IEventMiddleware

Ok, Whether subclasses that inherit from it need to change the class name and file name

IEventMiddleware

Ok, Whether subclasses that inherit from it need to change the class name and file name. @zhenlei520

Unified modification, but the original writing will not be deleted for the time being, the expired feature will be added, and how to use the new usage will be specified, and it will be deleted when the official version 1.0.0 is released

I think accepting a break change would make things easier

ok,let me confirm step.  @zhenlei520 @doddgu

add new class file,it's name is :IEventMiddleware
modifcation all inhert from IMiddleware subclass,change to inhert IEventMiddleware? or say: all inhert from IMiddleware subclass unchange?

ok,let me confirm step. @zhenlei520 @doddgu

add new class file,it's name is :IEventMiddleware
modifcation all inhert from IMiddleware subclass,change to inhert IEventMiddleware? or say: all inhert from IMiddleware subclass unchange?

wherther need change all subclass's name?  example: ValidatorMiddleware=> ValidatorEventMiddleware

ok,let me confirm step. @zhenlei520 @doddgu

add new class file,it's name is :IEventMiddleware
modifcation all inhert from IMiddleware subclass,change to inhert IEventMiddleware? or say: all inhert from IMiddleware subclass unchange?

If you want compatibility to 1.0, you can inherit IMiddleware and IEventMiddleware

wherther need change all subclass's name? example: ValidatorMiddleware=> ValidatorEventMiddleware

Yes.

ok,let me confirm step. @zhenlei520 @doddgu

add new class file,it's name is :IEventMiddleware
modifcation all inhert from IMiddleware subclass,change to inhert IEventMiddleware? or say: all inhert from IMiddleware subclass unchange?

If you want compatibility to 1.0, you can inherit IMiddleware and IEventMiddleware

wherther need change all subclass's name? example: ValidatorMiddleware=> ValidatorEventMiddleware

Yes.

i suggest  direct editor 'IMiddleware',put it changed to 'IEventMiddelware'.   Because compatibility to 1.0 version requires edit  a lot  of class file.   also need add 'IEventMiddelware' support to service. it's a lot of work. If it is only modified on the original basis, the workload is less.

I agree.
But, you need to provide a break change upgrade guide.
@zhenlei520 Is there upgrade guide?

It is possible, but you need to introduce this modification in the upgrade guide and in-process event documentation to avoid confusion caused by renaming
Document address: https://github.com/masastack/MASA.Docs
@MapleWithoutWords


KeyError: 'values' - verbrauch_data = api.verbrauch(date_to_fetch)['values']

Since yesterday, 2023-11-09, the applications crashes with following error:
Traceback (most recent call last):
File "/app/./main.py", line 68, in <module>
     verbrauch_data = api.verbrauch(date_to_fetch)['values']
KeyError: 'values'

Did they made any changes to the API?

Same issue here:

I can login/authenticate
I get the "Zaehlpunktnummer"
but e.g. sending the URL _https://api.wstw.at/WN_SMART_METER_PORTAL_APIB2C/1.0/messdaten/zaehlpunkt/xx(=meinZählpunkt)/verbrauchRaw?dateFrom=2023-11-07T00%3A00%3A00.00Z&dateTo=2023-11-08T00%3A00%3A00.00Z&granularity=DAY_
does give you a "404" !

Help wanted!

Analysis: It seems they have changed the API slightly
For the GET-requests the original URL was like:
https://api.wstw.at/WN_SMART_METER_PORTAL_API_B2C/1.0/messdaten/**zaehlpunkt**/nnn(=meinZählpunkt)/verbrauchRaw? ......
note the part "messdaten/zaehlpunkt/AT001......"  <-- there is the string "zaehlpunkt"
If I am correct, they have replaced "zaehlpunkt" with your actual customer number (should be part of the profile ?)
the new URL format is:
https://api.wstw.at/gateway/WN_SMART_METER_PORTAL_API_B2C/1.0/messdaten/**1200888888**/AT0010000000000000001000001111111/verbrauch? .......
I will try to do further tests on my side on the weekend
best regards
Gerhard
BTW: this is not my actual customer number and zaehlpunkt :-)

My Solution: in "vienna_smartmeter.py" I defined
   def customerid(self):
        """Returns 'geschaeftspartner' = CustomerID for currently logged in user."""
        return self._call_api_wn("w/user/profile")["defaultGeschaeftspartnerRegistration"]["geschaeftspartner"]

and in "_call_api_wstw" I did insert 2 lines before making the request (self.session.request)
   if endpoint != "zaehlpunkte":                                # if the endpoint is NOT just "zaehlpunkte"
        url = url.replace("zaehlpunkt", self.customerid())      # we substitute the string "zaehlpunkt" with our customer-id
    response = self.session.request(method, url, headers=headers, json=data)

As you can see: I just replace "zaehlpunkt" with the customerid .... maybe there are more elegant ways - but works for me.
best regards
Gerhard

Can confirm that this patch works

@ghhausi Awesome! Maybe do a pull request? 👍
@platysma FYI

a probably more elegant solution would be to fill in the customerid in the methods verbrauch and verbrauch_raw directly.
using the customerid method from @ghhausi
replace
endpoint = "messdaten/zaehlpunkt/{}/verbrauchRaw".format(zaehlpunkt)
with
endpoint = "messdaten/{0}/{1}/verbrauchRaw".format(self.customerid(),zaehlpunkt)
But I can't test this currently as the smartmeter-webportal is down for maintainance.

The api is back online. Made a PR with a fix. Please try it.

Doesn't work with mine.
my app:
# Import vienna-smartmeter module
from vienna_smartmeter import Smartmeter
from datetime import datetime, timedelta

# Smartmeter Credentials
username = '###'
password = '###'

api = Smartmeter(username, password)
print(api.profil()) # working

date_from_string = "2023-11-01T03:00:00.000000"
date_from = datetime.strptime(date_from_string, "%Y-%m-%dT%H:%M:%S.%f")

print(api.verbrauch_raw(date_from=date_from))

Output:
{'quarter-hour-opt-in': False, 'statistics': {'maximum': 0, 'minimum': 0, 'average': 0}}

Your code works for me.
How did you install the new version?
I did it using
pip install 'vienna-smartmeter @ git+https://github.com/fleinze/vienna-smartmeter.git'
Also, I received such an output when the wstw-api was still offline today. Maybe there is still an error in the api, you should try again in a few hours.


⚠ Bofrost.de has degraded performance

In fcf0ecf, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1341 ms

Resolved: Bofrost.de performance has improved in 23ec9f7.


ListOf(Int32).index_ is not implemented

See failing test test_Int32_index in types_test.py.

what do you expect here? this doesn't work in regular python either
x = [1, 2, 3]
x[[0]]

throws an exception.  Numpy allows this but not regular lists...

I thought you had asked me to raise this as an issue. I will remove it.


Incompatiable with html-preview package

When html-preview function is on, hide it then,  open  the setting  panel, part of the preview just display as a background underneath the setting tabs on the left.It's cool in someway, but odd indeed
Could you please fix that? thanks!

which html-preview package are you referring to? I'm getting more than a few search results :smile:

I should apologize first, I haven't described clearly, html-preview is just one of the chain.I tested it again in case of coincidence,here is what I have done to meet the bug:

Involved packages:atom-html-preview by webBoxio, ** maximize panes**  by santip

open a html file or something else.
press ctrl-shitf-H to active html-preview ,editor split into two panes one each side. html file on the left while preview is on the right.
press ctrl-shift-Enter to maximize the html file pane ,preview pane hid automatically.
open the setting pane by shortcuts ,the hidden preview appears on the left accidently.

I'm not sure if I have stated clearly ,anyway
Thank you for your time!

The Ctrl+Shift+Enter did nothing for me. But after resizing the HTML preview pane I could still see the settings menu shadow on top of it. After some fiddling, this is what I got:

Ok, I modified the shortcuts, It works for me .It actives   maximize panes(by santip)


CI should check for properly formatted wgsl

What problem does this solve or what need does it fill?
This is an example of a file which is not formatted properly. It has trailing whitespace.
crates\bevy_ui\src\render\ui.wgsl
What solution would you like?
There should be a step in the CI job which checks that wgsl files are formatted properly.
What alternative(s) have you considered?
None
Additional context
I noticed a random whitespace change in this PR caused by unformatted code being merged.

What tool should we use to do this check?

wgsl-analyzer has a bin crate to format: wgslfmt https://github.com/wgsl-analyzer/wgsl-analyzer/blob/main/crates/wgslfmt/src/main.rs
see https://github.com/wgsl-analyzer/wgsl-analyzer/issues/67#issuecomment-1789388437 for an example in how to install and use it

The problem with that tool is that it isn't meant to be used like cargo fmt. It requires passing in every file name.

Also, the tool seems dead. Pasting my comment from discord:
wgsl-analyzer looks dead. Should we fork it and bring it into the bevyengine org? There are issues that need to be fixed which block #15470/#15449.
If you think it sounds like a good idea, I'm willing to adopt it.

This is blocked on improvements to wgsl-analyzer, which I'll work on over time.


Fix .webp SrcSet Creation Filter

Pull Request: Fix .webp SrcSet Creation Filter
Description
This PR addresses an issue with the filter for creating .webp srcsets introduced in PR #2698. The original filter incorrectly searched for extensions with a leading period (e.g., ".jpg" and ".png"). As a result, no matches were found, preventing the srcset from being added to any figures. This occurred because the split operation removes the period from the file extensions.
Changes Made

Updated the filter to search for file extensions without the leading period (e.g., "jpg" and "png").
Ensured that the resource sets are now correctly created for the respective file extensions, allowing the .webp srcset to function as intended.

Impact
With this change, responsive images will now correctly generate their srcsets for the appropriate file types, improving image loading performance and supporting better responsiveness.
Fixes

Fixes #2777

Please review the changes and let me know if there are any further adjustments needed. Thank you!

^^ This - I came to fix this same issue, and found someone already got to it! simple fix.

Just missing the prettier fix.

Just missing the prettier fix.

I don't know whether prettier is requesting this because I only changed one line. But nevertheless, it's applied now.


dbt 1.6 - ERROR    X validation errors for ManifestV10

Describe the bug
Now that dbt 1.6 is fully released, we are seeing validation errors for manifests from that version. The validation is using a beta version of the manifest schema.
Users will see:
ERROR    X validation errors for ManifestV10
Need to update dbt-artifacts-parser to support the official V10 manifest schema. Issue here: https://github.com/yu-iskw/dbt-artifacts-parser/issues/54
Other context for the future:

We're using dbt-artifacts-parser to access dbt manifests as python objects vs. dicts

this isn't totally necessary and may be causing more trouble than it's worth

Is it worth forking dbt-artifacts-parser and owning that dependency ourselves?

Is it worth forking dbt-artifacts-parser and owning that dependency ourselves?

I think if we're considering that we should just drop the package entirely. It's nice to use objects vs. dictionary.get() everywhere but not needed

Without diving into the details, is it viable to say "1.6 is not supported," which would buy us some time? I'm assuming most dbt users are not actually on 1.6.
Sorry if i'm misunderstanding the problem / possible solution.
cc @sungchun12

@dlawin I recommend we drop it because this becomes a single point of failure outside our control each time dbt upgrades in version. We've already had dozens of conversations around this since 1.6 released.
@leoebfolsom  "1.6 is not supported" feels sloppy on our part. We should say we're actively working on it and fix it as a default, especially since this tool is so closely tied to dbt usage.

PR here https://github.com/yu-iskw/dbt-artifacts-parser/pull/55

this becomes a single point of failure outside our control each time dbt upgrades in version. We've already had dozens of conversations around this since 1.6 released.

@sungchun12  While this is true the point of failure stems from dbt itself
The package protects us from dbt by throwing validation errors when there are unexpected manifest schemas, with that said, the fields we depend on in the manifest seem pretty stable.

@dlawin I understand that. I guess the more fundamental question to ask is that are the tradeoffs better in using this dependency for those validation errors already built for us and having to micromanage this dependency each time dbt upgrades the manifest in a new version each quarter/patch?
My intuition goes towards the tradeoffs aren't worth it, but Dan you're more intimate with this than me, so can help me understand those tradeoffs in detail?


fix: UNMET PEER DEPENDENCY<EMAIL_ADDRESS>

+--<EMAIL_ADDRESS>| --<EMAIL_ADDRESS>-- UNMET PEER DEPENDENCY<EMAIL_ADDRESS>

ahh sorry, npm versioning rules are wierd

Current coverage is 86.14% (diff: 100%)

Merging #356 into master will not change coverage

@@             master       #356   diff @@
==========================================
  Files             4          4
  Lines           332        332
  Methods          68         68
  Messages          0          0
  Branches         71         71
==========================================
  Hits            286        286
  Misses           46         46
  Partials          0          0

Powered by Codecov. Last update c8494f3...2173df7

Closing in favor of #349 which will land shortly


[SPARK-12976][SQL] Add LazilyGenerateOrdering and use it for RangePartitioner of Exchange.

Add LazilyGenerateOrdering to support generated ordering for RangePartitioner of Exchange instead of InterpretedOrdering.

Test build #49975 has started for PR 10894 at commit 624927a.

Test build #49975 has finished for PR 10894 at commit 624927a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class LazilyGenerateOrdering(val ordering: Seq[SortOrder]) extends Ordering[InternalRow]

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49975/
Test PASSed.

Cool! Are there any remaining usages of InterpretedOrdering after this patch?

Yes, there are some places to use it to interpret (eval()) expressions like MaxOf, MinOf, LessThan, etc. or it is used as a fallback of GenerateOrdering in SparkPlan.

Test build #50956 has started for PR 10894 at commit 99597b8.

Test build #50956 has finished for PR 10894 at commit 99597b8.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50956/
Test FAILed.

Test build #50957 has started for PR 10894 at commit 644c861.

Test build #50957 has finished for PR 10894 at commit 644c861.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50957/
Test PASSed.

Merged build finished. Test PASSed.

Test build #51167 has started for PR 10894 at commit 7151a73.

Test build #51167 has finished for PR 10894 at commit 7151a73.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51167/
Test PASSed.

I spot one other place where we might be able to use this in TakeOrderedAndProject.execute(). There's also a usage in TakeOrderedAndProjectNode, but I'm not sure if we want to update that one.
Also, one minor naming suggestion: I'd probably call this LazilyGeneratedOrdering, with a 'd', since it is an ordering rather than a factory for making them.
Aside from those two minor changes, this looks good to me. Sorry for missing those suggestions on the first review pass.

I renamed LazilyGenerateOrdering to LazilyGeneratedOrdering and modify TakeOrderedAndProject to use it.
And I found TakeOrderedAndProjectNode could use GenerateOrdering.generate() (not lazy) and UnsafeProjection.create() (not interpret).
It seems other LocalNodes are using generated projections or predicates.
Should we use them for TakeOrderedAndProjectNode?

Test build #51225 has started for PR 10894 at commit 3684b70.

Test build #51225 has finished for PR 10894 at commit 3684b70.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51225/
Test FAILed.

Test build #51232 has started for PR 10894 at commit bd1a53c.

Test build #51232 has finished for PR 10894 at commit bd1a53c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51232/
Test PASSed.

Merged build finished. Test PASSed.

This looks good to me, so I'm going to merge it into master. Thanks @ueshin!
Regarding TakeOrderedAndProjectNode, I think we should handle that change in a separate patch since it doesn't seem directly related to LazilyGeneratedOrdering. Feel free to open a new JIRA and PR for that change.

@JoshRosen Thank you for merging this.
I opened a new issue for TakeOrderedAndProjectNode and PRed at #11230.


feat: Enhance language dropdown ordering

Feature Description

[ ] Rearrange the dropdown menu in alphabetical order.
[ ] Standardize the notation of languages.
a. I think the format "Language, (English Notation)" is good, as is the case with many languages. e.g., 日本語 (Japanese).

Background
The current order of the dropdown menu in cncf/glossary appears as shown in the image below. It does not seem to be sorted in any particular order. (This might be the order in which the Localize team was initialized)

Additionally, in some languages, the notation seems to be slightly different. (Português, Italiano, and Deutsch appear to have a slightly different notation.)
Related Issue/PR
In kubernetes/website repo, a similar PR was created:

https://github.com/kubernetes/website/pull/44869

duplicate #2480


schema: Add a range limit to Swappiness

https://github.com/opencontainers/runtime-spec/blame/master/specs-go/config.go#L296
Signed-off-by: zhouhao<EMAIL_ADDRESS>

On Mon, Apr 24, 2017 at 10:48:20PM -0700, Zhou Hao wrote:

https://github.com/opencontainers/runtime-spec/blame/master/specs-go/config.go#L296

The Go comments are not normative 1, so you cannot use them as
grounds for validation.  We could bring that range limit into the
normative Markdown of config-linux.md, but the current maintainer
position seems to be against carrying kernel limits (e.g. see #755).
On the other hand, the spec isn't particularly consistent about
whether it carries kernel limits (e.g. it has them in [2,3,4,…]), so
maybe maintainers will want to carry this one.

I don't think they should be in spec, I've opened https://github.com/opencontainers/runtime-spec/pull/780 to remove the remain ones.

Closing since we are getting rid of tracking kernel limits.


Add assert.dom().isChecked() assertion

That would be great!

I guess this is a duplicate of https://github.com/simplabs/qunit-dom/issues/38, but more focused on a single assertion, so I'll keep this open. PRs obviously welcome 😉

🤦♀ oops, did not check the issues well enough. Sorry about that!

no worries, the other issue doesn't have a particularly expressive title 😉


Issuing service API for issuing a new certificate on demand

It's currently only possible to add/remove certificates from txacme by touching/deleting certificate files in the certificate directory (or adding/removing from some other store implementation).
It's technically possible to directly store empty certificates in an ICertificateStore and then call the AcmeIssuingService._check_certs() but that is messy and may be prone to race conditions with the timed invocation of the same method.
It should be possible to safely add and remove certificates from txacme.
A further iteration on this is perhaps the ability to modify certificates, e.g. to change (or as it is described in the certbot client, "expand") the SANs (#37).

I opened #77 to cover the "remove a cert" case separately.


vendor: github.com/containerd/platforms v0.2.1

Remove hcsshim import from repo

un-exports GetOsVersion

Update windows matcher to not compare empty os version

full diff: https://github.com/containerd/platforms/compare/v0.2.0...v0.2.1

v0.2.1 tag is up https://github.com/containerd/platforms/releases/tag/v0.2.1

Updated to v0.2.1 and moved out of draft 👍


\kern fixes, \hskip support, \TeX, \LaTeX, \KaTeX

This PR does a lot of cleanup on \kern-related functions, and defines \TeX and \LaTeX macros (fixing #224).  It also proposes a possible replacement to \KaTeX (needs discussion).
In more detail:

Move \kern, \mkern into functions directory (and use new defineFuction interface)

Add \hskip, \mskip support (but without supporting plus/minus -- this should be a future issue)

Properly separate \kern, \hskip from \mkern, \mskip.
(The former work in both modes, and don't support mu units.
The latter work only in math mode and only support mu units.)

Render horizontal spacing commands (such as \kern) using MathML <mspace>.
Previously this was rendered as an empty <mrow> with a TODO by @kevinbarabash to figure it out.

Implement \TeX macro identical to LaTeX's definition using \raisebox and \kern.

Implement \LaTeX macro roughly identical to LaTeX's definition (using \raisebox instead of a vertical box with a fill to get top alignment).  Comments welcome, but I think this is a pretty good solution for now.

I tried implementing a new \KaTeX logo using a similar macro (now that we have all these features), instead of the large body of code in src/functions/katex.js and static/katex.less.  It's currently defined as \katex so that you can check out this branch and compare the two:

I'd like to suggest that this definition replace the old \KaTeX, to reduce code complexity, but this definitely needs discussion.  Note the spacing is a little different, because it seemed natural to me to more closely match the spacing in \LaTeX.  Feel free to disagree!
Incidentally, this is also a step toward implementing \KaTeX in LaTeX, as mentioned in #793.  (The existing definition would be challenging to port.)

Cool!

Properly separate \kern, \hskip from \mkern, \mskip.
(The former work in both modes, and don't support mu units.
The latter work only in math mode and only support mu units.)

I think we'll need to bump the version again as this may break some math that was working for people.

Render horizontal spacing commands (such as \kern) using MathML .
Previously this was rendered as an empty  with a TODO by @kevinbarabash to figure it out.

Thanks for figuring this out.

Implement \LaTeX macro roughly identical to LaTeX's definition (using \raisebox instead of a vertical box with a fill to get top alignment). Comments welcome, but I think this is a pretty good solution for now.

Cool.  I think some people will be pretty stoked about this.

I tried implementing a new \KaTeX logo using a similar macro (now that we have all these features), instead of the large body of code in src/functions/katex.js and static/katex.less. It's currently defined as \katex so that you can check out this branch and compare the two:

I think the new \katex implementation is closer to the \LaTeX rendering, especially the kerning between the a and the T.  It might look better if there was more space between the K and a.  Looking at \LaTeX, there's an even amount of space between the L and the a and the a and the T.

Properly separate \kern, \hskip from \mkern, \mskip.

I have, so far, purposely omitted \mkern from the function support page because I thought it might break. Once the behavior of \mkern and \mskip is more reliable, I'll put them into the documentation.

I need to respond about the other issues (thanks for the review, @kevinbarabash!), but one question I have is whether the errors on misuse of \kern (especially) and \mkern (less so) should be turned into warnings.  We don't really care about the separation for our purposes, it's just to encourage people to use LaTeX correctly, so maybe a warning is better?  (We could still bump the version number, but maybe unnecessary to break existing code that uses \kern{..mu}.)

I addressed @kevinbarabash comments (thanks!) and switched from errors to warning when misusing mu units/mkern (and rebased to remove conflict).  I kept it an error to use \mkern/\mskip outside math mode, because this seems natural, and previously no kern/skip commands worked in text mode at all.

Here's a texdiff for \LaTeX and \TeX (red is KaTeX, green is LaTeX):

It appears to me that the A is slightly smaller in KaTeX, which causes the horizontal shift of the L.  I don't understand why; I confirmed that the A is in a font size exactly 70% of the T, and LaTeX says the same in its rendering:
\hbox(0.0+0.0)x15.0
\OT1/cmr/m/n/10 L
\kern -3.6
\vbox(6.83331+0.0)x5.90282, glue set 2.04997fil
.\hbox(4.78334+0.0)x5.90282
..\OT1/cmr/m/n/7 A
.\glue 0.0 plus 1.0fil minus 1.0fil
\kern -1.49994
\OT1/cmr/m/n/10 T
\kern -1.66702
\hbox(6.83331+0.0)x6.80557, shifted 2.15277
.\OT1/cmr/m/n/10 E
\kern -1.25
\OT1/cmr/m/n/10 X
spacefactor 1000

I also notice that there's some difference between Chrome vs. Firefox rendering (notice height of A):
 vs.
Personally I'd be OK with leaving it as is, as I'm running out of ideas (and time) for how to improve it, it matches the definition, and the rendering is reasonably close now.
I've committed this as a new screenshot test so that it can be explored later.  I also switched over \KaTeX to the new macro, and deleted the old function code.  So now the latest choice for the KaTeX logo is in the screenshot test.

@edemaine I'm not sure why all of the fonts screenshot tests are failing. 😕

OK, should be all fixed now.

@edemaine thanks for investing and fixing the issue.


Updated sites.js.

Hello!
https://cadmican.neocities.org/
The webring icon is located on the bottom of the page (located in the footer), its ID is #webring. If you wish any modifications to it please let me know!

Just in case you weren't aware, it looks like your essay links are leading to either a blank html page, or not linking to anything.

Yes, acknowledged that! I've added another disclaimer that these essays are not finished until yet and thus empty. This should be "fixed" until the end of next week!
Until now, all essays are not finished and thus are not available on this site right now. These are placeholders as this site is still work in progress.


feat: nextjs publishing fulltext search

Added full text search capability to nextjs publishing. The search works on both the titles and the bodies of notes.
Currently, the search data is exported with the nextjs export, and it gets fetched and loaded the first time the user clicks on the search bar. The fusejs index is provided through a generalized React hook, and new hooks can be added easily. As an example, I added a hook to generate the index on the client side, which is not currently used.
The following images were cropped vertically, but not horizontally.
Desktop (1440p)

Mobile (Samsung Note 10+)

Simulated responsive views

Added:

Preview for the matching note text, with context, with matching part highlighted
Matching parts in the title highlighted

Wide:

Narrow:

The "..." parts only get added if the matching part isn't at the start or end of the note.
One caveat is only the first matching part of the note text is displayed. I also implemented a version that displays all matching parts, but it ends up being very confusing in my opinion:


Axis label formatting

Hello,
I have a couple questions about formatting the axes.
For X axis, is there a way to display arbitrary category tick labels?  For instance, I would like to display text instead of number for categories (like 'America' 'Europe' 'Afria' etc. rather than 1, 2, 3.
For Y axis, how can I format the ticks as currency, for example?
I like the simplicity of the library and the output.  Thanks for writing it!

if i only want to increase font size of bar chart? what would be the class?

Would be nice to have the font size as a property for axes objects.

I modified the font of my x-axis with this css tag:
.c3 svg g g.c3-axis.c3-axis-x {
font: 20px sans-serif;
}

This modifies the tick fonts too :(


Need to fix the navbar line not getting align with the "li"

Need to fix it too

Can you assign me this issue?

Can you assign me this issue?

Sure go for it 👍🏻


Query cost module

Codecov Report

Merging #4 (6de4bd6) into master (ffc2ab8) will not change coverage.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##              master        #4   +/-   ##
===========================================
  Coverage     100.00%   100.00%
- Complexity        13        29   +16
===========================================
  Files              2         4    +2
  Lines             27        62   +35
===========================================
+ Hits              27        62   +35

Impacted Files
Coverage Δ

src/Exception/MaximalQueryCostWasReached.php
100.00% <100.00%> (ø)

src/QueryCostModule.php
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffc2ab8...6de4bd6. Read the comment docs.


Writing frame immediately after window creation doesn't work?

Sorry if I've missed something in the docs about this. The following example produces a window filled with black pixels even though I'm passing it white pixels:
use minifb::{WindowOptions, Window};

fn main() {

    let input_size: usize = 64;
    let mut input_buffer = vec![16777215; input_size.pow(2)]; // white buffer
    let mut window = Window::new("Input", input_size, input_size, WindowOptions::default()).unwrap();
    window.update_with_buffer(&input_buffer).unwrap();
    std::thread::sleep(std::time::Duration::from_millis(10000));

}

If I add a 100ms sleep right after creating the window (before the update_with_buffer line), then it works fine. So it seems like the window needs some "setup time"?

I'm not sure what the problem is here but it's likely related to something in the OS. Minifb mostly calls to the OS level functions (such as StretchBlit on Windows) I'm not sure if the window needs to be updated a couple of time before it takes effect.

For the record, I'm running Ubuntu 18.04. I don't have access to windows or mac to see if it's just isolated to Ubuntu/Linux. It's easy to work around anyway so it's not a big problem. By the way - while I've got you: thanks so much for the very handy crate! It's probably about third in my most-used crates after rand and rayon.

Yeah. My point was just that it likely differs depending on how each operating system handles updates to the window so it's hard to say a strict guideline here. Personally I have never used minifb for doing just one update but have a loop that shows something until end. Usually you have something that is outlined in the example such as
  while window.is_open() {
    window.update_with_buffer(...);

Also thanks! It's great to see that people find it useful :) Makes the work worth it!


Login via Auth0 does not persist

After login into Backstage application via Auth0 auth-provider, the login session does not persist.
Expected Behavior
Post login via Auth0 or any Oauth2.0 based authentication provider, the local storage should have saved session details. Subsequent refreshes or new tabs should not request user login again.
Current Behavior
After login to Backstage via Auth0, the session details do not get saved in the local storage. This leads to repeated user login requests upon refresh/new tab.
All the information required for the session schema is available. The error is in the session local storage.

Possible Solution
https://github.com/backstage/backstage/blob/master/packages/core-app-api/src/lib/AuthSessionManager/RefreshingAuthSessionManager.ts does not have a saveSession implementation.
Taking Github as an example because this error is not present in login via Github.
Github is using a mux for session storage comprised of AuthSessionStore https://github.com/backstage/backstage/blob/master/packages/core-app-api/src/lib/AuthSessionManager/AuthSessionStore.ts#L132 that has a saveSession.
A possible fix could be to have a similar saveSession available for OAuth2.
Steps to Reproduce

Setup Auth0 configuration by providing clientId, clientSecret and domain in app-config.yaml file.
Follow https://backstage.io/docs/api/deprecations#generic-auth-api-refs to setup auth0

Add the following code to apis.ts file:
export const acmeAuthApiRef: ApiRef<
  OAuthApi &
  ProfileInfoApi &
  BackstageIdentityApi &
  SessionApi
  > = createApiRef({
  id: 'internal.auth.acme',
});

  createApiFactory({
    api: acmeAuthApiRef,
    deps: {
      discoveryApi: discoveryApiRef,
      oauthRequestApi: oauthRequestApiRef,
      configApi: configApiRef,
    },
    factory: ({ discoveryApi, oauthRequestApi, configApi }) =>
      OAuth2.create({
        discoveryApi,
        oauthRequestApi,
        provider: {
          id: 'auth0',
          title: 'Auth0',
          icon: () => null,
        },
        defaultScopes: ['openid', 'email', 'profile'],
        environment: configApi.getOptionalString('auth.environment'),
      }),
  })

Add the following to identityProvider.ts file
  {
    id: 'auth0-auth-provider',
    title: 'Auth0',
    message: 'Sign In using Auth0',
    apiRef: acmeAuthApiRef,
  },

Add the following to App.tsx
    <Route
      path="/settings"
      element={
        <UserSettingsPage
          providerSettings={
            <ProviderSettingsItem
              title="AUTH0"
              description="Provides sign-in via Auth0"
              apiRef={acmeAuthApiRef}
              icon={Star}
            />
          }
        />
      }
    />

yarn start both the backend and frontend
Login via Auth0
Inspect the local session storage. It will not have any session stored.
Press refresh/open another Backstage app tab. The login does not persist.

Context
Inability to configure login via Auth0 or OAuth2 based auth-providers.
Your Environment

NodeJS Version (v14): v14.18.3
Output of yarn backstage-cli info command on my Backstage App

OS:   Darwin 20.6.0 - darwin/x64
node: v14.18.3
yarn: 1.22.1
cli:  0.10.2 (local)

Dependencies:
  @backstage/catalog-client       0.3.19
  @backstage/core-app-api         0.1.24
  @backstage/core-components      0.3.3, 0.4.2, 0.5.0, 0.6.1, 0.7.6
  @backstage/core-plugin-api      0.1.13, 0.2.2
  @backstage/plugin-catalog-react 0.4.6, 0.5.2
✨  Done in 1.25s.

Thank you for reporting!
The OAuth2 -based providers don't and shouldn't store sessions in local storage. They're stored in a cookie scoped to the auth provider on the backend instead. This is quite different that the GitHub auth implementation, which doesn't work the same way.
Still wondering what's going on here though. Do you see any cookies if you navigate to <backend.baseUrl>/api/auth/auth0-auth-provider/handler/frame?
Are there any errors shown in the logs or anything else that's indicating that something's up?

Oh okay, thanks for your response @Rugvip :)
Do you see any cookies if you navigate to <backend.baseUrl>/api/auth/auth0/handler/frame?

Yes I can see a cookie being set here auth0-nonce
However, there is no cookie on the frontend. Shouldnt it require a cookie to create front-end session, so that it comes with the next request to the backend/application? Why is the cookie being set on backend domain?

Are there any errors shown in the logs or anything else that's indicating that something's up?

No, couldnt find any error in the logs

Also, after successful login when I press refresh the following call is made:
<backend.baseUrl>/api/auth/auth0/refresh?optional&env=development
with the following error response:
"Refresh token is not supported for provider auth0"

Oh that would explain it 😁
Looks like it's explicitly disabled, and has been since it was added. I'm not really seeing anything in the original PR that makes it look like that was intentional, so I'm assuming it's just a bug.
Do you have a setup where you can try disabling it easily? I think it's enough that we just verify that refresh works as expected and we can prolly then get rid of the flag

Okay, I tried to enable refresh explicitly to see if refresh is working but then received the following error response:
"Refresh failed; caused by InputError: Missing session cookie"

Alright, might simply be the case that auth0 doesn't support refresh in the default setup?

I checked my application settings on Auth0 to see if I missed out this configuration. But I have set it up:

Hmm, if you log the refresh token that's being passed through here, do you see it, or is it empty? Wanna figure out if this is an issue on the passport/auth0 side or if it's something going wrong after that.

I checked, its being passed as undefined

Hmm, alright, so at least we know that it's not being lost somewhere then. It may be that Auth0 requires a particular scope to be passed for refresh to be allowed. You could try adding 'offline_access' here, like for example the OneLogin provider does.

@Rugvip Thanks alot for your help! This seems to have worked. Not just by adding 'offline_access' here owing to deprecation but also by the following change at apis.ts file.
  createApiFactory({
    api: acmeAuthApiRef,
    deps: {
      discoveryApi: discoveryApiRef,
      oauthRequestApi: oauthRequestApiRef,
      configApi: configApiRef,
    },
    factory: ({ discoveryApi, oauthRequestApi, configApi }) =>
      OAuth2.create({
        discoveryApi,
        oauthRequestApi,
        provider: {
          id: 'auth0',
          title: 'Auth0',
          icon: () => null,
        },
        defaultScopes: ['openid', 'email', 'profile', '**offline_access**'],
        environment: configApi.getOptionalString('auth.environment'),
      }),
  })

We still need to consider removing the explicit disableRefresh setting.
Maybe we can allow it via config like https://github.com/backstage/backstage/blob/f92294b0047eac83d1346649b937d09d4dc217e3/plugins/auth-backend/src/providers/oauth2/provider.ts#L247 ?

Nice! :grin:
I think it's alright to have refresh always enabled. Do you want to create a PR that applies these fixes? 🙏
Good find on the change to apis.ts as well, forgot we did that 😅

Thanks alot! Sure, I ll create a PR

Hi @Rugvip, I am facing similar issues to this but with the Okta Auth provider. I can confirm that the okta-refresh-token cookie is set on the backend API domain (which is different to our frontend domain) after logging in. This doesn't seem to be sent when the browser is refreshed which is resulting in the user being redirected to the signin page and popup. Upon sign in popup, the login then proceeds automatically. Do you have any thoughts on what might be happening here?

@R-cen alright, let's dig into the refresh request in that case, since that seems to be the trouble. Does the refresh request fail and in that case in what way?
The request looks something like this, although ofc okta instead of github:

You should be able to see a okta-refresh-token cookie as part of the request headers of that request. If it's not there then it would seem the cookie isn't being set correctly and there might be a misconfigured baseUrl or callbackUrl somewhere. If you see the cookie in the client-side request headers, then I'm wondering whether it arrives in the server-side request? You could also try to execute a refresh request from outside the browser via curl or similar to debug from there.

thanks @Rugvip, for now we have updated both the backend and frontend to be served on the same domain instead which resolves the issue we were seeing

Going to close this, but if anyone else runs into this please let us know and we can keep debugging!

@Rugvip  I know it's been a while since this closed, but I'm currently running into this issue with OKTA.  I can login fine, and if I hit /api/auth/okta/handler/frame?env=myenv I can see the okta-nonce cookie.  But the session persistence is not there.  Once it tries to call /api/auth/okta/refresh?optional&env=myenv the okta-refresh-token cookie is not there and I get the error message of "name":"AuthenticationError","message":"Invalid X-Requested-With header" with a status code of 401. offline_access is setup.  Any other ideas?  You started to mention baseUrl or callbackUrl, but looks like previous person figured it out before you got too far there.

I figured out the issue.  offline_access scope is probably the answer if you're having issues with session duration.  The OKTA provider out of the box has offline_access as a default scope so that's probably not the issue if you're using OKTA.  However, if you're using OKTA you have to make sure it is setup at the auth server and app server level.  That was my issue.  I had it setup at the app server level, but didn't at the auth server and never really saw any error or warning so the app server wasn't really doing anything as that is supposed to be a subset(or all) of the scopes the auth server has.  Hope this helps someone.

Hi @carlossmartini I have the same issue you had with the "Invalid X-Requested-With header" and status code of 401 on refresh.
I added already the prompt: auto as from here  and the Okta team confirmed that they assigned the following scopes to the refresh: groups, openid, profile, email, offline_access.
Can you provide some more details In order for me to understand what I am missing?


replace serde_macros by serde_derive

serde_macros is being deprecated in favor of serde_derive[0].
As a result, rustfix does not build on nighlty anymore[1].
This commit replaces serde_macros by serde_derive.
[0]
https://users.rust-lang.org/t/serde-transitioning-to-macros-1-1/7437/5
[1] https://github.com/serde-rs/serde/issues/606#issuecomment-258648508
Unfortunately, this is not enough:
error[E0277]: the trait bound `rustfix::diagnostics::Diagnostic: serde::de::Deserialize` is not satisfied
  --> src/main.rs:78:26
   |
78 |         .flat_map(|line| serde_json::from_str::<Diagnostic>(line))
   |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `serde::de::Deserialize` is not implemented for `rustfix::diagnostics::Diagnostic`
   |
   = note: required by `serde_json::from_str`

I dont' get this error because rustfix::diagnostics::Diagnostic does derive Deserialize: https://github.com/killercup/rustfix/blob/master/src/diagnostics.rs#L5
Feel free to close in favor of a working PR, otherwise, I'll try to finish this tonight.

Thanks! I think you only need minimal changes to make this work :)
You need to load serde_derive and enable the new macro feature. Reading https://serde.rs/codegen.html, I think you just need to replace the first few lines in lib.rs with this:
#![feature(proc_macro)]

#[macro_use]
extern crate serde_derive;

extern crate serde_json;

If you want to, you can also try to make this compile on stable Rust (as a different PR maybe).
By the way, can you also change the nightly's date to a more recent one in .travis.yml?

Thank you for your help! It now builds on the latest nightly:
$ rustc --version
rustc 1.14.0-nightly (cae6ab1c4 2016-11-05)

Thanks again!


Create a tutorial to deploy a simple Hello Email app on Microsoft Azure with Node.js using this SDK

Create a tutorial that explains how to create a simple Hello Email app and deploy to Microsoft Azure.
The example should utilize best practices for storing credentials. By the end of the tutorial, the user should be able to send an email using their deployed app.
Please add the tutorial to USE_CASES.md.

I would like to give this one a shot. ✋🏻

Any update @thinkingserious ?

@SarasArya,
Please add it here: https://github.com/sendgrid/sendgrid-nodejs/blob/master/packages/mail/USE_CASES.md
Thank you!
With Best Regards,
Elmer

Hi Elmer
Should the tutorial reiterate the steps mentioned in the doc? Or just skip to the end, where I assume the app is already deployed and running on Azure? and going to a particular endpoint actually sends an email.
I don't see a point in reiterating the same steps.

It should be sufficient to reference the Azure docs for the steps specified there.

@thinkingserious  #490 done


Add initial CircleCI config

This is not a very sophisticated build system, but it does get the ball rolling for adding testing for more of the projects.
I had to include #85 in this branch as the bin/eftest script is broken on master (thanks to me).

I'm going to think about this one a little. While I want to have CI for Edge, I want to make sure this doesn't make it harder for users to use circle ci themselves.
For example if someone scraps this file and starts over in their repo, what happens with git conflicts? I think git does the right thing (oh, you deleted it back their, so discard conflicts on this file). But I want to make sure.
Ideally this would be something that users could build upon. For example, I love the aggregate deps.edn cache, that's a great thing to reuse!  Although, it doesn't do clj for all those deps.edn files, so the .m2 folder will probably need some additional population first.

For example if someone scraps this file and starts over in their repo, what happens with git conflicts? I think git does the right thing (oh, you deleted it back their, so discard conflicts on this file). But I want to make sure.

This is a good point, I hadn't considered this.

Although, it doesn't do clj for all those deps.edn files, so the .m2 folder will probably need some additional population first.

My approach in the past when using Leiningen with CI testing has been to just let the tests populate the ~/.m2 cache. The boot time to download the dependencies was sometimes quite slow, and often there were dynamic dependencies added after the fact anyway.
Using clj, you don't have as much boot time, so it should be fine to download the dependencies in a separate step. However we still add eftest as a dynamic dependency in the script. I'm not sure how much it buys you pre-downloading the artifacts, but I don't really mind either way.


How to achieve full screen on the web?

Hi! I've never done it myself but have you tried configuring your screen with WindowMode::Fullscreen. I don't know whether it's supported in browsers or not though. Anyway, it seems like this question is related to Bevy itself, so I think you can find more help in its repository or Discord.
Let me know if there's anything that you believe bevy_egui can assist with for fullscreen windows.

@mvlabat The problem is fixed, now the problem is on winit. In bevy, there is also this problem. In winit, the window in the web represents a canvas instance, and winit does not manage the Window object. When the Window changes, winit cannot know it. So I used web_sys to monitor the resize event of the Window by myself. A system is added to bevy to handle the resize event of Window, and change the size of the canvas when the resize event of Window occurs. Also, at the beginning, use the width and height of the Window to set the size of the canvas.
Also, on the web, winit handles WindowDescriptor.mode badly!

Here is my key code:
use std::sync::{
    atomic::{AtomicBool, Ordering},
    Arc,
};

use bevy::prelude::*;
use wasm_bindgen::{prelude::*, JsCast};

pub struct WebPlugin;

impl Plugin for WebPlugin {
    fn build(&self, app: &mut App) {
        // window resize event flag
        let window_resize_evt = WindowResizeEvent::default();

        let mut window_resize_evt0 = window_resize_evt.clone();

        // 用于处理 Window 的 resize event
        let window_resize_handle = WindowResizeHandle {
            listener: Closure::wrap(Box::new(move || {
                window_resize_evt0.set();
                log::info!("resize!");
            }) as Box<dyn FnMut()>),
        };

        app.insert_resource(window_resize_evt);
        app.insert_resource(window_resize_handle);
        app.add_startup_system(window_maxium);
        app.add_startup_system(window_resize_register);
        app.add_system(window_resize_nofity);
    }
}

// 初始化时, 设置canvas最大化
fn window_maxium(mut windows: ResMut<Windows>) {
    let window = web_sys::window().expect("no global `window` exists");
    let width: f32 = window.inner_width().unwrap().as_f64().unwrap() as f32;
    let height: f32 = window.inner_height().unwrap().as_f64().unwrap() as f32;
    if let Some(window) = windows.get_primary_mut() {
        window.set_resolution(width, height);
        log::info!("set_resolution({width}, {height})");
    }
}

/// 注册 Window 的 resize事件
fn window_resize_register(window_resize_handle: Res<WindowResizeHandle<dyn FnMut()>>) {
    let window = web_sys::window().expect("Failed to obtain window");
    window.set_onresize(Some(window_resize_handle.listener.as_ref().unchecked_ref()));
}

fn window_resize_nofity(
    mut window_resize_evt: ResMut<WindowResizeEvent>,
    windows: ResMut<Windows>,
) {
    if window_resize_evt.get() {
        window_maxium(windows);
        window_resize_evt.clear();
    }
}

pub struct WindowResizeHandle<T: ?Sized> {
    pub listener: Closure<T>,
}

unsafe impl<T: ?Sized> Send for WindowResizeHandle<T> {}
unsafe impl<T: ?Sized> Sync for WindowResizeHandle<T> {}

#[derive(Default, Clone)]
pub struct WindowResizeEvent {
    happened: Arc<AtomicBool>,
}

impl WindowResizeEvent {
    pub fn set(&mut self) {
        self.happened.store(true, Ordering::SeqCst);
    }
    pub fn clear(&mut self) {
        self.happened.store(false, Ordering::SeqCst);
    }
    pub fn get(&mut self) -> bool {
        self.happened.load(Ordering::SeqCst)
    }
}

Oh, I assumed you needed a real fullscreen mode (like on youtube).
Thanks for sharing your example of how you were able to accomplish this.
This is how I also resize the canvas in my pet project (it's not yet updated to Bevy 0.6 though, I don't know if anything's changed in this regard): https://github.com/mvlabat/muddle-run/blob/fbdba0ce0e32ef1b7763be568b910b499bdd70f6/bins/web_client/src/lib.rs#L15
It seems like it didn't require setting a window resolution manually. But, as I said, it might be outdated.


Renamed win_ffi to win_32

Not pulling until a new adapter has been created. Please pull my latest changes before you're moving forward with the new adapter creation...


yazi_floating_window_winblend behaving oddly with icons

When enabling this configuration point yazi behaves oddly with icons that are wider than monospace allows.
I've tested this in kitty and alacritty.  Both behave slightly differently.

My yazi.nvim
return {
	"mikavilpas/yazi.nvim",
	event = "VeryLazy",
	dependencies = {
		{ "nvim-lua/plenary.nvim", lazy = true },
	},
	keys = {
		-- 👇 in this section, choose your own keymappings!
		{
			"<leader>_",
			function()
				require("yazi").yazi()
			end,
			desc = "Open the file manager",
		},
		{
			-- Open in the current working directory
			"<leader>e",
			function()
				require("yazi").yazi(nil, vim.fn.getcwd())
			end,
			desc = "[E]xecute Yazi File Tree 󰙅 ",
		},
	},
	opts = {
		-- if you want to open yazi instead of netrw, see below for more info
		open_for_directories = true,
		floating_window_scaling_factor = 0.85,
		-- Bugged?
		-- yazi_floating_window_winblend = 20,
	},
}

I doubt this has any effect, but my plugin list is rather small and fully curated by myself.  Not a bulky distro like NVChad or LazyVIM.

In kitty:

Kitty is broken by making the icons with text behind the overflow, half sized so there is no overflow.
Kitty resumes normally after disabling.
In Alacritty:

As you can see, it behaves differently because the right portion of icons that have text behind them allow the text to bleed through as though the far right end of the icon is transparent, or the 'bg color'.

Alacritty also resumes normal behavior after commenting.
This behavior does not exist outside the confines of neovim loading the plugin.  There are no transparency options (I'm aware of) so raw terminal instances of Yazi both in Zellij and outside the influence of a muxer behave normally.
In terms of where to begin looking, this interesting comment was made on the Yazi discord that may explain this:

EDIT/UPDATE:
Installed Foot terminal and it works beautifully with the feature enabled:

If you'd like a minimal reproduction, please point me at exactly what that entails and I'll get it done for you.
Thank you for the exceedingly handy plugin!

Thanks for the bug report! This looks like an interesting adventure to be sure...
I tried your settings out but wasn't able to reproduce this on wezterm. Maybe it's terminal specific, but I would like to rule out any chance of conflicting plugins first.
Could you follow the instructions here to create a minimal reproduction? https://github.com/mikavilpas/yazi.nvim/blob/main/documentation/reproducing-issues.md

Minimal Repro steps:

Downloaded repro.lua from https://github.com/mikavilpas/yazi.nvim/blob/main/documentation/reproducing-issues.md to my ~/Downloads directory
ran nvim -u ~/Downloads/repro.lua -c "lua require('lazy').update()"
Lazy loaded minimalistic plugin set:

set yazi_floating_window_winblend = 20 (or 25) in opts block
restart nvim with the above command to be thorough
using Foot terminal, as before works normally
using Kitty terminal, as before cuts icon size in half

Alacritty terminal, as before allows background image to bleed all the way through right hand portion of icons

The only change I made to the repro.lua was enabling that single feature.  No other plugins were loaded.  If I could get wezterm to work on my box I'd test it, and I'm happy to test any other terminals (Wayland friendly) if it will help.
As directed, my repro.lua

---@module "yazi"
---@module "lazy"

-- DO NOT change the paths and don't remove the colorscheme
local root = vim.fn.fnamemodify('./.repro', ':p')

-- set stdpaths to use .repro
for _, name in ipairs({ 'config', 'data', 'state', 'cache' }) do
  vim.env[('XDG_%s_HOME'):format(name:upper())] = root .. '/' .. name
end

-- bootstrap lazy
local lazypath = root .. '/plugins/lazy.nvim'
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    'git',
    'clone',
    '--filter=blob:none',
    'https://github.com/folke/lazy.nvim.git',
    lazypath,
  })
end
vim.opt.runtimepath:prepend(lazypath)
vim.g.mapleader = ' '

-- install the following plugins
---@type LazySpec
local plugins = {
  'folke/tokyonight.nvim',
  {
    'mikavilpas/yazi.nvim',
    event = 'VeryLazy',
    keys = {
      {
        -- 👇 choose your own keymapping
        '<leader>fy',
        function()
          require('yazi').yazi()
        end,

        { desc = 'Open the file manager' },
      },
    },
    ---@type YaziConfig
    opts = {
      open_for_directories = false,
      yazi_floating_window_winblend = 20,

    },
  },
}
require('lazy').setup(plugins, {
  root = root .. '/plugins',
})

vim.cmd.colorscheme('tokyonight')
-- add anything e

Have a fantastic weekend!

Do you think the kitty issue might be this? https://yazi-rs.github.io/docs/faq#why-icons-shrink
Looks like the recommendation/workaround is to use a flavor. Can you test this out?
If you're currently not using a flavor, there is an example of the change you need to make here https://github.com/mikavilpas/dotfiles/commit/abbf921f5c85087da7e272f7f1dce5f26ab97c91
basically it's just installing the flavor and then configuring yazi to use it.

I think the issue that was mentioned might have been this https://github.com/sxyazi/yazi/issues/906

This does not seem to solve the problem for me.

If you'll remember I tried this in foot, kitty & Alacritty.  Alacritty was also exhibiting a slight variation of the oddity.  Refactoring from a theme to a flavor sadly did not seem to make any changes/fix.
Added the following table to my yazi.lua neovim plugin:
	{
		"yazi-rs/flavors",
		name = "yazi-flavor-catppuccin-macchiato",
		lazy = true,
		build = function(spec)
			require("yazi.plugin").build_flavor(spec, {
				sub_dir = "catppuccin-macchiato.yazi",
			})
		end,
	},

Pointed my yazi.toml away from theme and onto flavor.  I won't bore you with that diff.  The colors and appearance changed from my everforest theme to catppuccin so I know it hooked up but the icons when yazi_floating_window_winblend = 15 (or other arbitrary integers around 15)

New interesting thing!
This is an oil floating window, with winblend enabled.

Notice the exact same issue?  I do.
AKA this happens as well with yazi and yazi.nvim completely removed from the scenario.

That's very interesting indeed. I looked at the oil.nvim repo as well as neovim's repo, but couldn't find any leads in about 20 minutes of investigation.
It might be some neovim bug, but I'm not really sure. Kind of out of ideas.

That's very interesting indeed. I looked at the oil.nvim repo as well as neovim's repo, but couldn't find any leads in about 20 minutes of investigation.
It might be some neovim bug, but I'm not really sure. Kind of out of ideas.

Yeah that's why I wanted to get you the update.  I don't think it's ANYTHING within your control as a plugin developer OR a yazi user.  Looks like any winblend with icons in the ecosystem seems to behave the same way.


[FEATURE] Add comment syntax for new languages on Dodona

Is this the only needed change in https://github.com/thepieterdc/dodona-plugin-vscode/blob/master/src/util/comments.ts?
const exclamation = (text: string) => `<!-- ${text} -->`;

html: exclamation,
sql: dashes,

If so, I'm willing to make a PR for it.

That should be enough to make it work.