StringBuilder ensure capacity overkill

https://github.com/konsoletyper/teavm/blob/master/classlib/src/main/java/org/teavm/classlib/java/lang/TAbstractStringBuilder.java#L580
wouldnt that allocate too much capacity if the request capacity is very big?

I would recommend to go through all the classes that use on demand capacity expansion.


Radix Sort JavaScript

💥 Proposal
I would like to add radix sort in javascript
Have you read the Contributing Guidelines on Pull Requests?
Yes

/assign


[Feature]: new page under devops for oracle cloud

Is your feature request related to a problem? Please describe.
https://www.nsccbpit.tech/devops
create the page for the subject, connect it to the card.
On the page provide all information and resources related to the concerned topic such as tutorials, documentation, youtube channels, articles, courses etc.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response

I want to work on this issue assign me :)

@vaibhavshah03 go ahead

@vaibhavshah03 are you still working on this issue

can i work on this?

@pradeeptosarkar , Please assign me this issue; I have successfully resolved similar ones before.

go ahead @sagarbee


StateTransitionEvent

Objective

Make it possible to react to arbitrary state changes
this will be useful regardless of the other changes to states currently being discussed

Solution

added StateTransitionEvent<S> struct
previously, this would have been impossible:

#[derive(States, Eq, PartialEq, Hash, Copy, Clone, Default)]
enum MyState {
  #[default]
  Foo,
  Bar(MySubState),
}

enum MySubState {
  Spam,
  Eggs,
}

app.add_system(Update, on_enter_bar);

fn on_enter_bar(trans: EventReader<StateTransition<MyState>>){
  for (befoare, after) in trans.read() {
    match before, after {
      MyState::Foo, MyState::Bar(_) => info!("detected transition foo => bar");
      _, _ => ();
    }
  }
}

Changelog

Added

StateTransitionEvent<S> - Fired on state changes of S

Migration Guide
N/A no breaking changes

just making sure y'all aren't waiting for me to do something

I know I'm a bit late on this, but @SIGSTACKFAULT could you maybe give #10334 a look. I think it also does cover your use case and in addition to that it allows to react to arbitrary state changes at the "right" time (in between enter/exit).


No specs found after running SpecRunner

hi
i just started to write my very first script using jasmine , this is my AddressBookSpec.js file:
describe('Address Book', Function(){
it ('should be able to add a contact', function(){
var addressBook = new addressBook();
thisContact = new contact();
addressBook.addContact(thisContact);
expect(addressBook.getContact(0)).toBe(thisContact);
});
});
im trying to learn jasmine sing udemy courses and want to see the error the instructor got in specrunner in my browser , but after running specrunner on browser i have nothing there but just this:

Can anybody please help with that ?
Thanks

hello Afsoonsh,
I solved the proble by adding this:
//= require YOUR-FOLDER/AddresBook.js
at beginning of the file.
Lmk

Hi there
May be its too late .. but I had the same problem using Gedit for edit  AddressBookSpec.js.
The solution is set up in Gedit Preferences checkbox 'Insert spaces instead of tabs'


About limitation of Azure Firewall.

https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#azure-firewall-limits
Rule limits | 10,000 unique source/destinations in network rules
About above limitation, the meaning is not clear. Could you add the example about DNAT, Network and Application rule in order to count the number of combination ?
E,g:
rule collection1
rule1    src ip : <IP_ADDRESS>/24 src port: any    dst: <IP_ADDRESS>/24  dst port:22  tcp allow
rule1    src ip : <IP_ADDRESS>/24 src port: any    dst: <IP_ADDRESS>/24  dst port:23  tcp allow
The number of combination counts is 4 because source address is two and destination address is two.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 36f49a06-90b1-cbe1-3853-2c38277b79f8
Version Independent ID: 02ff6f6c-3ef6-b48f-060d-b290419e0aba
Content: Azure subscription limits and quotas - Azure Resource Manager
Content Source: articles/azure-resource-manager/management/azure-subscription-service-limits.md
Service: azure-resource-manager
Sub-service: management
GitHub Login: @tfitzmac
Microsoft Alias: tomfitz

Thanks for your feedback @hisasima .
Assigning to content author  @tfitzmac for review.

The article was updated to include both network and application rules. Thanks @vhorne
#please-close


is the file type from the post data?

from what i can see the file type is from the post data which can be spoofed?
why no use mime_content_type?

@pixelpaulaus mime_content_type is deprecated in new PHP versions


Add backward compatible API typings

Issue related: https://github.com/kmagiera/react-native-reanimated/issues/138

Looks good to me 👍

Only thing I would change is backwardCompatibleWrapper => BackwardCompatibleWrapper. After that merge and publish?

nice, thanks @henrikra
ping @kmagiera

Thanks @henrikra for the review. And thanks @osdnk for this PR. Merging in


gitkraken: 4.2.2 -> 5.0.1

Motivation for this change
Update gitkraken to 5.0.1
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

fixed typo in commit msg


Added few more data structure questions.

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)

Closes #XXXXX

Hi @Gautam-Hegde !
It looks like when you resolved the merge conflict you accidently deleted another contributors quiz questions.
When merge conflicts come up it is important to add your quiz questions below the recently added ones.
If you need help resolving merge conflicts then you can make a comment on the PR here and we can resolve it for you.
I went ahead and added those questions back into your branch.
Thanks! 👍

Thank you for your quiz questions! 👍


Build failure because third_party/zlib references system wide zlib when building examples

Hello again :)
When writing a script to automate the build of gRPC via Travis-CI, I discovered that cmake references the system wide zlib to build third_party/zlib examples rather than the lib itlself. Here's how to reproduce the issue:
sudo apt-get remove zlib1g-dev
git clone -b v1.0.1 --depth 1 --recursive https://github.com/grpc/grpc.git
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Debug ../grpc
make -j 4

Scanning dependencies of target minigzip
[ 46%] Building C object third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o
/home/aurelien/test/grpc/third_party/zlib/test/minigzip.c:18:18: fatal error: zlib.h: No such file or directory
compilation terminated.
third_party/zlib/CMakeFiles/minigzip.dir/build.make:62: recipe for target 'third_party/zlib/CMakeFiles/minigzip.dir/test/minigzip.o' failed

If the system wide version of zlib is the same than third_party/zlib (1.2.8), then it's fine. Otherwise (like on Travis) we get an error.
For some reasons, it works fine when calling cmake directly on third_party/zlib, but it fails when invoking it from grpc/CMakeFiles.txt.
As I failed to force the use of third_party/zlib rather than the system wide one, I fixed the issue by removing the last lines of third_party/zlib/CMakeFiles.txt that want to build the examples.

The problem is due to an include_directories() call in the zlib's CMakeLists.txt. It does:
include_directories(${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_SOURCE_DIR})

When building zlib alone, we get:
-I/path/to/build-zlib -I/path/to/grpc/third_party/zlib

But when building grpc, it gives:
-I/path/to/build-grpc/third_party/zlib -I/path/to/grpc

Thus it cannot find the zlib.h file.
To fix the issue, the include_directories should be changed to:
include_directories(${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_SOURCE_DIR})

This issue is described in madler/zlib#218.


584 Customize membership request notifications

Hi @jorge-campo,
In accordance with our agreement, I am sending the PR after we have completed the two steps.
1 - Meet with Jorge to determine if the article provides the user with all the information they need.
2 - Worked with Emma to polish the English.
I would appreciate any suggestions you have. :)
Fabio

A screenshot has been added.


NHL fails when behind firewall  "urlopen error [Errno 1] _ssl.c:504: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure"

If I attempt to download from the URL:
https://www.nhl.com/lightning/video/jon-cooper-pc-041216/t-277437434/c-43457803
when NOT behind a firewall it works correctly.
However if I attempt the same URL from behind a firewall it is blocked (yes I can I view the video in a browser from behind firewall)
youtube-dl.py -U "https://www.nhl.com/lightning/video/jon-cooper-pc-041216/t-277437434/c-43457803"
youtube-dl is up-to-date (2016.04.24)
[nhl.com] 43457803: Downloading JSON metadata
ERROR: Unable to download JSON metadata: <urlopen error [Errno 1] _ssl.c:504: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure> (caused by URLError(SSLError(1, '_ssl.c:
504: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'),))
same error as https://github.com/rg3/youtube-dl/issues/5366
maybe?
Thanks
Ringo
Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.04.24. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X ] I've verified and I assure that I'm running youtube-dl 2016.04.24

Before submitting an issue make sure you have:

[X ] At least skimmed through README and most notably FAQ and BUGS sections
[X ] Searched the bugtracker for similar issues including closed ones
pls see: https://github.com/rg3/youtube-dl/issues/5366

What is the purpose of your issue?

[X ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
see above
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):
https://www.nhl.com/lightning/video/jon-cooper-pc-041216/t-277437434/c-43457803

Description of your issue, suggested solution and other information
see above

UPDATE: The URL works from behind firewall when running under Python 3.x  (fails with 2.x)
shall I close?


fix: broken dark mode screen caused by main html tag's height

Description
Dark mode's current behavior is broken. Whenever its children overflow its parent current height (which is of 100vh), white margins shows up. Changing the main tag style from height: 100vh to minHeight: 100vh fixes this, as the main tag will then obey its children height.
Screenshots

Before
After

Thanks @marcelovicentegc !


Separators do not work in iOS 9

They just don't show. ActionSheet.separatorView can be set as any color but they seem to default to the background color. However, they continue to work as intended on iOS 9.

Hi! Thanks for reporting this bug.
It seems to work for me on iOS 9, though. I changed separatorColor in the example project to red and got:


Make idiff quiet

I'm using idiff for automated testing, and it currently is too noisy for that use case. For me it would be preferable if idiff only outputed anything to stderr on failure.
I propose adding flags:
-q --quiet -  Don't output anything to stdout/stderr, user relies on return code. This one is easy.
-qd - Don't output anything unless diff fails. This one is a bit more complex, since you'll have to
output "Comparing a b" message before error messages.
Just making idiff always use stderr will be fine for me, but I think solution should be more generic.

I'm fine with -q meaning a quiet mode that outputs nothing for success, and only printing "FAILURE" to stderr if it fails (and setting return code, of course). Is that sufficient? Or do you truly need NO output to stderr?
Another easy solution is to write your own Python script that uses the OIIO Python bindings to do something like
#!/usr/bin/env python
import sys
import OpenImageIO as oiio
comp = oiio.ImageBufAlgo.compare (oiio.ImageBuf(sys.argv[1]), oiio.ImageBuf(sys.argv[2]))
if comp.nwarn > 0 or comp.nfail > 0 :
    exit (-1)
else :
    exit (0)

Simple and does exactly what you want.

I'm ok with output to stderr, but currently idiff outputs to stdout which is not that great.
Python script would be ok too, but I still think -q mode would be nice.

Proposed fix in #1231
Please give it a try on your end and see if the behavior is satisfactory.

Exactly what I need, thank you!
I wanted to submit PR today, and was really surprised that you've implemented it already.
Also, I see that some of the thing which used to be written to stdout are now written to stderr even without flag and I hope this does not break idiff for people who rely on parsing stdout.

Oh, it only took a few minutes.
I did redirect a couple things from stdout to stderr, but only error messages.

Sorry if I stepped on your toes by doing this. Sometimes it's hard to discern whether one of these filed issues is a request for me to do something, or a proposal that somebody else is about to do it.

Do you need this backported to 1.5?

No need, I'm on master.


how to feed a placeholder with the return values of tf.train.batch()

Code:
images, raw_images, labels = tf.train.batch(
[image, raw_image, label],
batch_size = batch_size,
num_threads = 1,
capacity = 4 * batch_size,
allow_smaller_final_batch = True)
return images, raw_images, labels
images, _, labels = load_batch(dataset, batch_size=batch_size)
sess.run(train_op, feed_dict={x: x_img, y_true: label})
Errors:
sess.run(train_op, feed_dict={x: images, y_true: labels})
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1074, in _run
raise TypeError('The value of a feed cannot be a tf.Tensor object. '
TypeError: The value of a feed cannot be a tf.Tensor object. Acceptable feed values include Python scalars, strings, lists, numpy ndarrays, or TensorHandles.

#!/usr/bin/python
#-- coding: utf-8 --
import tensorflow as tf
from tensorflow.contrib.framework.python.ops.variables import get_or_create_global_step
from tensorflow.python.platform import tf_logging as logging
import inception_preprocessing
from inception_resnet_v2 import inception_resnet_v2, inception_resnet_v2_arg_scope
import os
import time
slim = tf.contrib.slim
import numpy as np
#设置定量的GPU使用量
config  = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.9
设置最小的GPU使用量
"""
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
"""
#================ DATASET INFORMATION ======================
#State dataset directory where the tfrecord files are located
dataset_dir = '.'
#State where your log file is at. If it doesn't exist, create it.
log_dir = './log'
#State where your checkpoint file is
checkpoint_file = './inception_resnet_v2_2016_08_30.ckpt'
#State the image size you're resizing your images to. We will use the default inception size of 299.
img_width = 800
img_height = 600
file_pattern = 'estate_%s_*.tfrecord'
#State the number of classes to predict:
num_classes = 6
#State the labels file and read it
labels_file = './labels.txt'
labels = open(labels_file, 'r')
#Create a dictionary to refer each label to their string name
labels_to_name = {}
for line in labels:
label, string_name = line.split(':')
string_name = string_name[:-1] #Remove newline
labels_to_name[int(label)] = string_name
#Create the file pattern of your TFRecord files so that it could be recognized later on
file_pattern = 'estate_%s_*.tfrecord'
#Create a dictionary that will help people understand your dataset better. This is required by the Dataset class later.
items_to_descriptions = {
'image': 'A 3-channel RGB coloured real estate image that is either bathroom, bedroom, floorplan, kitchen, or livingroom, other.',
'label': 'A label that is as such -- 0:bathroom, 1:bedroom, 2:floorplan, 3:kitchen, 4:livingroom, 5:other'
}
#================= TRAINING INFORMATION ==================
#State the number of epochs to train
num_epochs = 1
#State your batch size
batch_size = 1
#Learning rate information and configuration (Up to you to experiment)
initial_learning_rate = 0.0002
learning_rate_decay_factor = 0.7
num_epochs_before_decay = 2
#iteration
total_iterations = 0
#============== DATASET LOADING ======================
#We now create a function that creates a Dataset class which will give us many TFRecord files to feed in the examples into a queue in parallel.
def get_split(split_name, dataset_dir, file_pattern=file_pattern, file_pattern_for_counting='estate'):
'''
Obtains the split - training or validation - to create a Dataset class for feeding the examples into a queue later on. This function will
set up the decoder and dataset information all into one Dataset class so that you can avoid the brute work later on.
Your file_pattern is very important in locating the files later.
INPUTS:
- split_name(str): 'train' or 'validation'. Used to get the correct data split of tfrecord files
- dataset_dir(str): the dataset directory where the tfrecord files are located
- file_pattern(str): the file name structure of the tfrecord files in order to get the correct data
- file_pattern_for_counting(str): the string name to identify your tfrecord files for counting

OUTPUTS:
- dataset (Dataset): A Dataset class object where we can read its various components for easier batch creation later.
'''

#First check whether the split_name is train or validation
if split_name not in ['train', 'validation']:
    raise ValueError('The split_name %s is not recognized. Please input either train or validation as the split_name' % (split_name))

#Create the full path for a general file_pattern to locate the tfrecord_files
file_pattern_path = os.path.join(dataset_dir, file_pattern % (split_name))

#Count the total number of examples in all of these shard
num_samples = 0
file_pattern_for_counting = file_pattern_for_counting + '_' + split_name
tfrecords_to_count = [os.path.join(dataset_dir, file) for file in os.listdir(dataset_dir) if file.startswith(file_pattern_for_counting)]
for tfrecord_file in tfrecords_to_count:
    for record in tf.python_io.tf_record_iterator(tfrecord_file):
        num_samples += 1

#Create a reader, which must be a TFRecord reader in this case
reader = tf.TFRecordReader

#Create the keys_to_features dictionary for the decoder
keys_to_features = {
  'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
  'image/format': tf.FixedLenFeature((), tf.string, default_value='jpg'),
  'image/class/label': tf.FixedLenFeature(
      [], tf.int64, default_value=tf.zeros([], dtype=tf.int64)),
}

#Create the items_to_handlers dictionary for the decoder.
items_to_handlers = {
'image': slim.tfexample_decoder.Image(),
'label': slim.tfexample_decoder.Tensor('image/class/label'),
}

#Start to create the decoder
decoder = slim.tfexample_decoder.TFExampleDecoder(keys_to_features, items_to_handlers)

#Create the labels_to_name file
labels_to_name_dict = labels_to_name

 #Actually create the dataset
#dataset 对象定义了数据集的文件位置，解码方式等元信息
dataset = slim.dataset.Dataset(
    data_sources = file_pattern_path,
    decoder = decoder,
    reader = reader,
    num_readers = 4,
    num_samples = num_samples,
    num_classes = num_classes,
    labels_to_name = labels_to_name_dict,
    items_to_descriptions = items_to_descriptions)
return dataset

def load_batch(dataset, batch_size, height=img_height, width=img_width, is_training=True):
'''
Loads a batch for training.
INPUTS:
- dataset(Dataset): a Dataset class object that is created from the get_split function
- batch_size(int): determines how big of a batch to train
- height(int): the height of the image to resize to during preprocessing
- width(int): the width of the image to resize to during preprocessing
- is_training(bool): to determine whether to perform a training or evaluation preprocessing

OUTPUTS:
- images(Tensor): a Tensor of the shape (batch_size, height, width, channels) that contain one batch of images
- labels(Tensor): the batch's labels with the shape (batch_size,) (requires one_hot_encoding).

'''
#First create the data_provider object
data_provider = slim.dataset_data_provider.DatasetDataProvider(
    dataset,
    common_queue_capacity = 24 + 3 * batch_size,
    common_queue_min = 24)

#Obtain the raw image using the get method
raw_image, label = data_provider.get(['image', 'label'])

#Perform the correct preprocessing for this image depending if it is training or evaluating
image = inception_preprocessing.preprocess_image(raw_image, height, width, is_training)

#As for the raw images, we just do a simple reshape to batch it up
raw_image = tf.expand_dims(raw_image, 0)
raw_image = tf.image.resize_nearest_neighbor(raw_image, [height, width])
#modify due to data type
raw_image = tf.cast(raw_image, tf.float32)
raw_image = tf.squeeze(raw_image)

#Batch up the image by enqueing the tensors internally in a FIFO queue and dequeueing many elements with tf.train.batch.
images, raw_images, labels = tf.train.batch(
    [image, raw_image, label],
    batch_size = batch_size,
    num_threads = 1,
    capacity = 4 * batch_size,
    allow_smaller_final_batch = True)

return images, raw_images, labels

sess = tf.Session()
    #Create the log directory here. Must be done here otherwise import will activate this unneededly.

if not os.path.exists(log_dir):
os.mkdir(log_dir)
#======================= TRAINING PROCESS =========================
#Now we start to construct the graph and build our model
tf.logging.set_verbosity(tf.logging.INFO) #Set the verbosity to INFO level
dataset = get_split('train', dataset_dir, file_pattern=file_pattern)
#First create the dataset and load one batch
x = tf.placeholder(tf.float32, shape=[None, img_height, img_width,3], name='x')
#y_true = tf.placeholder(tf.int32, shape=[None, num_classes], name='y_true')
y_true = tf.placeholder(tf.int32, shape=[num_classes], name='y_true')
#images = tf.reshape(x, [-1, 800, 600, 1])
#Know the number steps to take before decaying the learning rate and batches per epoch
num_batches_per_epoch = int(dataset.num_samples / batch_size)
num_steps_per_epoch = num_batches_per_epoch #Because one step is one batch processed
decay_steps = int(num_epochs_before_decay * num_steps_per_epoch)
#Create the model inference
with slim.arg_scope(inception_resnet_v2_arg_scope()):
logits, end_points = inception_resnet_v2(x, num_classes = dataset.num_classes, is_training = True)
y_pred = tf.nn.softmax(logits, name='y_pred')
#Define the scopes that you want to exclude for restoration
exclude = ['InceptionResnetV2/Logits', 'InceptionResnetV2/AuxLogits']
variables_to_restore = slim.get_variables_to_restore(exclude = exclude)
#Perform one-hot-encoding of the labels (Try one-hot-encoding within the load_batch function!)
one_hot_labels = slim.one_hot_encoding(y_true, dataset.num_classes)
#Performs the equivalent to tf.nn.sparse_softmax_cross_entropy_with_logits but enhanced with checks
loss = tf.losses.softmax_cross_entropy(onehot_labels = one_hot_labels, logits = logits)
total_loss = tf.losses.get_total_loss()    #obtain the regularization losses as well
#Create the global step for monitoring the learning_rate and training.
global_step = get_or_create_global_step()
#Define your exponentially decaying learning rate
lr = tf.train.exponential_decay(
learning_rate = initial_learning_rate,
global_step = global_step,
decay_steps = decay_steps,
decay_rate = learning_rate_decay_factor,
staircase = True)
#Now we can define the optimizer that takes on the learning rate
optimizer = tf.train.AdamOptimizer(learning_rate = lr)
#Create the train_op.
train_op = slim.learning.create_train_op(total_loss, optimizer)
#State the metrics that you want to predict. We get a predictions that is not one_hot_encoded.
predictions = tf.argmax(end_points['Predictions'], 1)
probabilities = end_points['Predictions']
accuracy, accuracy_update = tf.contrib.metrics.streaming_accuracy(predictions, y_true)
metrics_op = tf.group(accuracy_update, probabilities)
sess.run(tf.global_variables_initializer())
#Now finally create all the summaries you need to monitor and group them into one summary op.
tf.summary.scalar('losses/Total_Loss', total_loss)
tf.summary.scalar('accuracy', accuracy)
tf.summary.scalar('learning_rate', lr)
my_summary_op = tf.summary.merge_all()
#Now we need to create a training step function that runs both the train_op, metrics_op and updates the global_step concurrently.
def train_step(sess, train_op, global_step):
'''
Simply runs a session for the three arguments provided and gives a logging on the time elapsed for each global step
'''
#Check the time for each sess run
start_time = time.time()
total_loss, global_step_count, _ = sess.run([train_op, global_step, metrics_op])
time_elapsed = time.time() - start_time
#Run the logging to print some results
#if global_step_count % 10 == 0:
logging.info('global step %s: loss: %.4f (%.2f sec/step)', global_step_count, total_loss, time_elapsed)
return total_loss, global_step_count

#Now we create a saver function that actually restores the variables from a checkpoint file in a sess
saver = tf.train.Saver(variables_to_restore)
saver.restore(sess, checkpoint_file)
def train(num_iteration):
global total_iterations
for i in range(total_iterations,
               total_iterations + num_iteration):
    #如果是batch 的话可以增加一个循环控制
    images, _, labels = load_batch(dataset, batch_size=batch_size)
    sess.run(train_op, feed_dict={x: images, y_true: labels})
    saver.save(sess,  global_step = global_step)
total_iterations += num_iteration

train(num_iteration=24000)

#!/usr/bin/python
#-- coding: utf-8 --
import tensorflow as tf
from tensorflow.contrib.framework.python.ops.variables import get_or_create_global_step
from tensorflow.python.platform import tf_logging as logging
import inception_preprocessing
from inception_resnet_v2 import inception_resnet_v2, inception_resnet_v2_arg_scope
import os
import time
slim = tf.contrib.slim
import numpy as np
config  = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.9
"""
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
"""
#================ DATASET INFORMATION ======================
#State dataset directory where the tfrecord files are located
dataset_dir = '.'
#State where your log file is at. If it doesn't exist, create it.
log_dir = './log'
#State where your checkpoint file is
checkpoint_file = './inception_resnet_v2_2016_08_30.ckpt'
#State the image size you're resizing your images to. We will use the default inception size of 299.
img_width = 800
img_height = 600
file_pattern = 'estate_%s_*.tfrecord'
#State the number of classes to predict:
num_classes = 6
#State the labels file and read it
labels_file = './labels.txt'
labels = open(labels_file, 'r')
#Create a dictionary to refer each label to their string name
labels_to_name = {}
for line in labels:
label, string_name = line.split(':')
string_name = string_name[:-1] #Remove newline
labels_to_name[int(label)] = string_name
#Create the file pattern of your TFRecord files so that it could be recognized later on
file_pattern = 'estate_%s_*.tfrecord'
#Create a dictionary that will help people understand your dataset better. This is required by the Dataset class later.
items_to_descriptions = {
'image': 'A 3-channel RGB coloured real estate image that is either bathroom, bedroom, floorplan, kitchen, or livingroom, other.',
'label': 'A label that is as such -- 0:bathroom, 1:bedroom, 2:floorplan, 3:kitchen, 4:livingroom, 5:other'
}
#================= TRAINING INFORMATION ==================
#State the number of epochs to train
num_epochs = 1
#State your batch size
batch_size = 1
#Learning rate information and configuration (Up to you to experiment)
initial_learning_rate = 0.0002
learning_rate_decay_factor = 0.7
num_epochs_before_decay = 2
#iteration
total_iterations = 0
#============== DATASET LOADING ======================
#We now create a function that creates a Dataset class which will give us many TFRecord files to feed in the examples into a queue in parallel.
def get_split(split_name, dataset_dir, file_pattern=file_pattern, file_pattern_for_counting='estate'):
'''
Obtains the split - training or validation - to create a Dataset class for feeding the examples into a queue later on. This function will
set up the decoder and dataset information all into one Dataset class so that you can avoid the brute work later on.
Your file_pattern is very important in locating the files later.
INPUTS:
- split_name(str): 'train' or 'validation'. Used to get the correct data split of tfrecord files
- dataset_dir(str): the dataset directory where the tfrecord files are located
- file_pattern(str): the file name structure of the tfrecord files in order to get the correct data
- file_pattern_for_counting(str): the string name to identify your tfrecord files for counting

OUTPUTS:
- dataset (Dataset): A Dataset class object where we can read its various components for easier batch creation later.
'''

#First check whether the split_name is train or validation
if split_name not in ['train', 'validation']:
    raise ValueError('The split_name %s is not recognized. Please input either train or validation as the split_name' % (split_name))

#Create the full path for a general file_pattern to locate the tfrecord_files
file_pattern_path = os.path.join(dataset_dir, file_pattern % (split_name))

#Count the total number of examples in all of these shard
num_samples = 0
file_pattern_for_counting = file_pattern_for_counting + '_' + split_name
tfrecords_to_count = [os.path.join(dataset_dir, file) for file in os.listdir(dataset_dir) if file.startswith(file_pattern_for_counting)]
for tfrecord_file in tfrecords_to_count:
    for record in tf.python_io.tf_record_iterator(tfrecord_file):
        num_samples += 1

#Create a reader, which must be a TFRecord reader in this case
reader = tf.TFRecordReader

#Create the keys_to_features dictionary for the decoder
keys_to_features = {
  'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
  'image/format': tf.FixedLenFeature((), tf.string, default_value='jpg'),
  'image/class/label': tf.FixedLenFeature(
      [], tf.int64, default_value=tf.zeros([], dtype=tf.int64)),
}

#Create the items_to_handlers dictionary for the decoder.
items_to_handlers = {
'image': slim.tfexample_decoder.Image(),
'label': slim.tfexample_decoder.Tensor('image/class/label'),
}

#Start to create the decoder
decoder = slim.tfexample_decoder.TFExampleDecoder(keys_to_features, items_to_handlers)

#Create the labels_to_name file
labels_to_name_dict = labels_to_name

 #Actually create the dataset
dataset = slim.dataset.Dataset(
    data_sources = file_pattern_path,
    decoder = decoder,
    reader = reader,
    num_readers = 4,
    num_samples = num_samples,
    num_classes = num_classes,
    labels_to_name = labels_to_name_dict,
    items_to_descriptions = items_to_descriptions)
return dataset

def load_batch(dataset, batch_size, height=img_height, width=img_width, is_training=True):
'''
Loads a batch for training.
INPUTS:
- dataset(Dataset): a Dataset class object that is created from the get_split function
- batch_size(int): determines how big of a batch to train
- height(int): the height of the image to resize to during preprocessing
- width(int): the width of the image to resize to during preprocessing
- is_training(bool): to determine whether to perform a training or evaluation preprocessing

OUTPUTS:
- images(Tensor): a Tensor of the shape (batch_size, height, width, channels) that contain one batch of images
- labels(Tensor): the batch's labels with the shape (batch_size,) (requires one_hot_encoding).

'''
#First create the data_provider object
data_provider = slim.dataset_data_provider.DatasetDataProvider(
    dataset,
    common_queue_capacity = 24 + 3 * batch_size,
    common_queue_min = 24)

#Obtain the raw image using the get method
raw_image, label = data_provider.get(['image', 'label'])

#Perform the correct preprocessing for this image depending if it is training or evaluating
image = inception_preprocessing.preprocess_image(raw_image, height, width, is_training)

#As for the raw images, we just do a simple reshape to batch it up
raw_image = tf.expand_dims(raw_image, 0)
raw_image = tf.image.resize_nearest_neighbor(raw_image, [height, width])
#modify due to data type
raw_image = tf.cast(raw_image, tf.float32)
raw_image = tf.squeeze(raw_image)

#Batch up the image by enqueing the tensors internally in a FIFO queue and dequeueing many elements with tf.train.batch.
images, raw_images, labels = tf.train.batch(
    [image, raw_image, label],
    batch_size = batch_size,
    num_threads = 1,
    capacity = 4 * batch_size,
    allow_smaller_final_batch = True)

return images, raw_images, labels

sess = tf.Session()
    #Create the log directory here. Must be done here otherwise import will activate this unneededly.

if not os.path.exists(log_dir):
os.mkdir(log_dir)
#======================= TRAINING PROCESS =========================
#Now we start to construct the graph and build our model
tf.logging.set_verbosity(tf.logging.INFO) #Set the verbosity to INFO level
dataset = get_split('train', dataset_dir, file_pattern=file_pattern)
#First create the dataset and load one batch
x = tf.placeholder(tf.float32, shape=[None, img_height, img_width,3], name='x')
#y_true = tf.placeholder(tf.int32, shape=[None, num_classes], name='y_true')
y_true = tf.placeholder(tf.int32, shape=[num_classes], name='y_true')
#images = tf.reshape(x, [-1, 800, 600, 1])
#Know the number steps to take before decaying the learning rate and batches per epoch
num_batches_per_epoch = int(dataset.num_samples / batch_size)
num_steps_per_epoch = num_batches_per_epoch #Because one step is one batch processed
decay_steps = int(num_epochs_before_decay * num_steps_per_epoch)
#Create the model inference
with slim.arg_scope(inception_resnet_v2_arg_scope()):
logits, end_points = inception_resnet_v2(x, num_classes = dataset.num_classes, is_training = True)
y_pred = tf.nn.softmax(logits, name='y_pred')
#Define the scopes that you want to exclude for restoration
exclude = ['InceptionResnetV2/Logits', 'InceptionResnetV2/AuxLogits']
variables_to_restore = slim.get_variables_to_restore(exclude = exclude)
#Perform one-hot-encoding of the labels (Try one-hot-encoding within the load_batch function!)
one_hot_labels = slim.one_hot_encoding(y_true, dataset.num_classes)
#Performs the equivalent to tf.nn.sparse_softmax_cross_entropy_with_logits but enhanced with checks
loss = tf.losses.softmax_cross_entropy(onehot_labels = one_hot_labels, logits = logits)
total_loss = tf.losses.get_total_loss()    #obtain the regularization losses as well
#Create the global step for monitoring the learning_rate and training.
global_step = get_or_create_global_step()
#Define your exponentially decaying learning rate
lr = tf.train.exponential_decay(
learning_rate = initial_learning_rate,
global_step = global_step,
decay_steps = decay_steps,
decay_rate = learning_rate_decay_factor,
staircase = True)
#Now we can define the optimizer that takes on the learning rate
optimizer = tf.train.AdamOptimizer(learning_rate = lr)
#Create the train_op.
train_op = slim.learning.create_train_op(total_loss, optimizer)
#State the metrics that you want to predict. We get a predictions that is not one_hot_encoded.
predictions = tf.argmax(end_points['Predictions'], 1)
probabilities = end_points['Predictions']
accuracy, accuracy_update = tf.contrib.metrics.streaming_accuracy(predictions, y_true)
metrics_op = tf.group(accuracy_update, probabilities)
sess.run(tf.global_variables_initializer())
#Now finally create all the summaries you need to monitor and group them into one summary op.
tf.summary.scalar('losses/Total_Loss', total_loss)
tf.summary.scalar('accuracy', accuracy)
tf.summary.scalar('learning_rate', lr)
my_summary_op = tf.summary.merge_all()
#Now we need to create a training step function that runs both the train_op, metrics_op and updates the global_step concurrently.
def train_step(sess, train_op, global_step):
'''
Simply runs a session for the three arguments provided and gives a logging on the time elapsed for each global step
'''
#Check the time for each sess run
start_time = time.time()
total_loss, global_step_count, _ = sess.run([train_op, global_step, metrics_op])
time_elapsed = time.time() - start_time
#Run the logging to print some results
#if global_step_count % 10 == 0:
logging.info('global step %s: loss: %.4f (%.2f sec/step)', global_step_count, total_loss, time_elapsed)
return total_loss, global_step_count

#Now we create a saver function that actually restores the variables from a checkpoint file in a sess
saver = tf.train.Saver(variables_to_restore)
saver.restore(sess, checkpoint_file)
def train(num_iteration):
global total_iterations
for i in range(total_iterations,
               total_iterations + num_iteration):
    images, _, labels = load_batch(dataset, batch_size=batch_size)
    sess.run(train_op, feed_dict={x: images, y_true: labels})
    saver.save(sess,  global_step = global_step)
total_iterations += num_iteration

train(num_iteration=24000)

#!/usr/bin/python
#-- coding: utf-8 --
import tensorflow as tf
from tensorflow.contrib.framework.python.ops.variables import get_or_create_global_step
from tensorflow.python.platform import tf_logging as logging
import inception_preprocessing
from inception_resnet_v2 import inception_resnet_v2, inception_resnet_v2_arg_scope
import os
import time
slim = tf.contrib.slim
import numpy as np
#设置定量的GPU使用量
config  = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = 0.9
设置最小的GPU使用量
"""
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
"""
#================ DATASET INFORMATION ======================
#State dataset directory where the tfrecord files are located
dataset_dir = '.'
#State where your log file is at. If it doesn't exist, create it.
log_dir = './log'
#State where your checkpoint file is
checkpoint_file = './inception_resnet_v2_2016_08_30.ckpt'
#State the image size you're resizing your images to. We will use the default inception size of 299.
img_width = 800
img_height = 600
file_pattern = 'estate_%s_*.tfrecord'
#State the number of classes to predict:
num_classes = 6
#State the labels file and read it
labels_file = './labels.txt'
labels = open(labels_file, 'r')
#Create a dictionary to refer each label to their string name
labels_to_name = {}
for line in labels:
label, string_name = line.split(':')
string_name = string_name[:-1] #Remove newline
labels_to_name[int(label)] = string_name
#Create the file pattern of your TFRecord files so that it could be recognized later on
file_pattern = 'estate_%s_*.tfrecord'
#Create a dictionary that will help people understand your dataset better. This is required by the Dataset class later.
items_to_descriptions = {
'image': 'A 3-channel RGB coloured real estate image that is either bathroom, bedroom, floorplan, kitchen, or livingroom, other.',
'label': 'A label that is as such -- 0:bathroom, 1:bedroom, 2:floorplan, 3:kitchen, 4:livingroom, 5:other'
}
#================= TRAINING INFORMATION ==================
#State the number of epochs to train
num_epochs = 1
#State your batch size
batch_size = 1
#Learning rate information and configuration (Up to you to experiment)
initial_learning_rate = 0.0002
learning_rate_decay_factor = 0.7
num_epochs_before_decay = 2
#iteration
total_iterations = 0
#============== DATASET LOADING ======================
#We now create a function that creates a Dataset class which will give us many TFRecord files to feed in the examples into a queue in parallel.
def get_split(split_name, dataset_dir, file_pattern=file_pattern, file_pattern_for_counting='estate'):
'''
Obtains the split - training or validation - to create a Dataset class for feeding the examples into a queue later on. This function will
set up the decoder and dataset information all into one Dataset class so that you can avoid the brute work later on.
Your file_pattern is very important in locating the files later.
INPUTS:
- split_name(str): 'train' or 'validation'. Used to get the correct data split of tfrecord files
- dataset_dir(str): the dataset directory where the tfrecord files are located
- file_pattern(str): the file name structure of the tfrecord files in order to get the correct data
- file_pattern_for_counting(str): the string name to identify your tfrecord files for counting

OUTPUTS:
- dataset (Dataset): A Dataset class object where we can read its various components for easier batch creation later.
'''

#First check whether the split_name is train or validation
if split_name not in ['train', 'validation']:
    raise ValueError('The split_name %s is not recognized. Please input either train or validation as the split_name' % (split_name))

#Create the full path for a general file_pattern to locate the tfrecord_files
file_pattern_path = os.path.join(dataset_dir, file_pattern % (split_name))

#Count the total number of examples in all of these shard
num_samples = 0
file_pattern_for_counting = file_pattern_for_counting + '_' + split_name
tfrecords_to_count = [os.path.join(dataset_dir, file) for file in os.listdir(dataset_dir) if file.startswith(file_pattern_for_counting)]
for tfrecord_file in tfrecords_to_count:
    for record in tf.python_io.tf_record_iterator(tfrecord_file):
        num_samples += 1

#Create a reader, which must be a TFRecord reader in this case
reader = tf.TFRecordReader

#Create the keys_to_features dictionary for the decoder
keys_to_features = {
  'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
  'image/format': tf.FixedLenFeature((), tf.string, default_value='jpg'),
  'image/class/label': tf.FixedLenFeature(
      [], tf.int64, default_value=tf.zeros([], dtype=tf.int64)),
}

#Create the items_to_handlers dictionary for the decoder.
items_to_handlers = {
'image': slim.tfexample_decoder.Image(),
'label': slim.tfexample_decoder.Tensor('image/class/label'),
}

#Start to create the decoder
decoder = slim.tfexample_decoder.TFExampleDecoder(keys_to_features, items_to_handlers)

#Create the labels_to_name file
labels_to_name_dict = labels_to_name

 #Actually create the dataset
#dataset 对象定义了数据集的文件位置，解码方式等元信息
dataset = slim.dataset.Dataset(
    data_sources = file_pattern_path,
    decoder = decoder,
    reader = reader,
    num_readers = 4,
    num_samples = num_samples,
    num_classes = num_classes,
    labels_to_name = labels_to_name_dict,
    items_to_descriptions = items_to_descriptions)
return dataset

def load_batch(dataset, batch_size, height=img_height, width=img_width, is_training=True):
'''
Loads a batch for training.
INPUTS:
- dataset(Dataset): a Dataset class object that is created from the get_split function
- batch_size(int): determines how big of a batch to train
- height(int): the height of the image to resize to during preprocessing
- width(int): the width of the image to resize to during preprocessing
- is_training(bool): to determine whether to perform a training or evaluation preprocessing

OUTPUTS:
- images(Tensor): a Tensor of the shape (batch_size, height, width, channels) that contain one batch of images
- labels(Tensor): the batch's labels with the shape (batch_size,) (requires one_hot_encoding).

'''
#First create the data_provider object
data_provider = slim.dataset_data_provider.DatasetDataProvider(
    dataset,
    common_queue_capacity = 24 + 3 * batch_size,
    common_queue_min = 24)

#Obtain the raw image using the get method
raw_image, label = data_provider.get(['image', 'label'])

#Perform the correct preprocessing for this image depending if it is training or evaluating
image = inception_preprocessing.preprocess_image(raw_image, height, width, is_training)

#As for the raw images, we just do a simple reshape to batch it up
raw_image = tf.expand_dims(raw_image, 0)
raw_image = tf.image.resize_nearest_neighbor(raw_image, [height, width])
#modify due to data type
raw_image = tf.cast(raw_image, tf.float32)
raw_image = tf.squeeze(raw_image)

#Batch up the image by enqueing the tensors internally in a FIFO queue and dequeueing many elements with tf.train.batch.
images, raw_images, labels = tf.train.batch(
    [image, raw_image, label],
    batch_size = batch_size,
    num_threads = 1,
    capacity = 4 * batch_size,
    allow_smaller_final_batch = True)

return images, raw_images, labels

sess = tf.Session()
    #Create the log directory here. Must be done here otherwise import will activate this unneededly.

if not os.path.exists(log_dir):
os.mkdir(log_dir)
#======================= TRAINING PROCESS =========================
#Now we start to construct the graph and build our model
tf.logging.set_verbosity(tf.logging.INFO) #Set the verbosity to INFO level
dataset = get_split('train', dataset_dir, file_pattern=file_pattern)
#First create the dataset and load one batch
x = tf.placeholder(tf.float32, shape=[None, img_height, img_width,3], name='x')
#y_true = tf.placeholder(tf.int32, shape=[None, num_classes], name='y_true')
y_true = tf.placeholder(tf.int32, shape=[num_classes], name='y_true')
#images = tf.reshape(x, [-1, 800, 600, 1])
#Know the number steps to take before decaying the learning rate and batches per epoch
num_batches_per_epoch = int(dataset.num_samples / batch_size)
num_steps_per_epoch = num_batches_per_epoch #Because one step is one batch processed
decay_steps = int(num_epochs_before_decay * num_steps_per_epoch)
#Create the model inference
with slim.arg_scope(inception_resnet_v2_arg_scope()):
logits, end_points = inception_resnet_v2(x, num_classes = dataset.num_classes, is_training = True)
y_pred = tf.nn.softmax(logits, name='y_pred')
#Define the scopes that you want to exclude for restoration
exclude = ['InceptionResnetV2/Logits', 'InceptionResnetV2/AuxLogits']
variables_to_restore = slim.get_variables_to_restore(exclude = exclude)
#Perform one-hot-encoding of the labels (Try one-hot-encoding within the load_batch function!)
one_hot_labels = slim.one_hot_encoding(y_true, dataset.num_classes)
#Performs the equivalent to tf.nn.sparse_softmax_cross_entropy_with_logits but enhanced with checks
loss = tf.losses.softmax_cross_entropy(onehot_labels = one_hot_labels, logits = logits)
total_loss = tf.losses.get_total_loss()    #obtain the regularization losses as well
#Create the global step for monitoring the learning_rate and training.
global_step = get_or_create_global_step()
#Define your exponentially decaying learning rate
lr = tf.train.exponential_decay(
learning_rate = initial_learning_rate,
global_step = global_step,
decay_steps = decay_steps,
decay_rate = learning_rate_decay_factor,
staircase = True)
#Now we can define the optimizer that takes on the learning rate
optimizer = tf.train.AdamOptimizer(learning_rate = lr)
#Create the train_op.
train_op = slim.learning.create_train_op(total_loss, optimizer)
#State the metrics that you want to predict. We get a predictions that is not one_hot_encoded.
predictions = tf.argmax(end_points['Predictions'], 1)
probabilities = end_points['Predictions']
accuracy, accuracy_update = tf.contrib.metrics.streaming_accuracy(predictions, y_true)
metrics_op = tf.group(accuracy_update, probabilities)
sess.run(tf.global_variables_initializer())
#Now finally create all the summaries you need to monitor and group them into one summary op.
tf.summary.scalar('losses/Total_Loss', total_loss)
tf.summary.scalar('accuracy', accuracy)
tf.summary.scalar('learning_rate', lr)
my_summary_op = tf.summary.merge_all()
#Now we need to create a training step function that runs both the train_op, metrics_op and updates the global_step concurrently.
def train_step(sess, train_op, global_step):
'''
Simply runs a session for the three arguments provided and gives a logging on the time elapsed for each global step
'''
#Check the time for each sess run
start_time = time.time()
total_loss, global_step_count, _ = sess.run([train_op, global_step, metrics_op])
time_elapsed = time.time() - start_time
#Run the logging to print some results
#if global_step_count % 10 == 0:
logging.info('global step %s: loss: %.4f (%.2f sec/step)', global_step_count, total_loss, time_elapsed)
return total_loss, global_step_count

#Now we create a saver function that actually restores the variables from a checkpoint file in a sess
saver = tf.train.Saver(variables_to_restore)
saver.restore(sess, checkpoint_file)
def train(num_iteration):
global total_iterations
for i in range(total_iterations,
               total_iterations + num_iteration):
    #如果是batch 的话可以增加一个循环控制
    images, _, labels = load_batch(dataset, batch_size=batch_size)
    sess.run(train_op, feed_dict={x: images, y_true: labels})
    saver.save(sess,  global_step = global_step)
total_iterations += num_iteration

train(num_iteration=24000)


[DOCS] Add suggested tags

Description
Suggested tags in linkfree profiles
Screenshots
No response
Additional information
We can have this docs

Could you elaborate on this. I didn't get it.

We can programmatically calculate the popular tags and then show them to people

@krshkun is that different from #3383 ?

Oh yes.


chore: add comments documenting Storage and AuthStorage

Working to implement the Storage interfaces, I found myself puzzled over what I could count on for TokenRequest beyond it's minimal interface.  It's minimal interface isn't rich enough for me to implement CreateAccessToken.   I've tried to address that with a comment, but a change in the API would be better (yeah,I know that would break backwards compatibility; sometimes that needs to happen).
Are my comments accurate?    What flow results in creating an access token after decoding a JWT?
Thanks
Note: I'm building a federated identity server so requests to my OP will trigger an outbound request as an OIDC RP or as a SAML SP depending on the SSO setup for that particular user.  After I get things working, I expect to try to make a change so that the OP /callback can be combined into the success handlers for the client authentication callbacks and thus avoid an intra-server redirect.

sorry for the late response and thank you very much for the continuous work on documentation and small fixes i've made some comments and additions to yours
BTW I'm open to changes to the interface. We've already created a v2 pre-release because we needed a way to dynamically get the issuer based on the host in ZITADEL (https://github.com/zitadel/oidc/tree/dynamic-issuer)

Great!
I'm not yet confident enough to make the changes to the interface.  My suggestion here would be to split up these two methods into ones that take more specific parameters and drop the op.TokenRequest type.


Fix "Policy contains a statement with one or more invalid principals."

When attaching policy to the CMK KMS key, the following error occurs:
"Policy contains a statement with one or more invalid principals."
The invalid principal is AWSServiceRoleForAutoScaling Role, which is not present on new AWS accounts which never had ASGs deployed on.

I do believe I have encountered this issue on eksctl as well, so we may just add a prerequisite in the documentation instead of actively checking and remediating.


Documentation Update for Key Vault Config

[Enter feedback here]
In .NET core 3.1 , i am able to read from key vault even if it disabled secerts only the disabled secrets are skipped and hence documenation can be updated to reflect in .Net core 3.1 it no longer fail to read configuration if there are disabled secrets.
Section containing troubleshooting and disabled/expired secrets should be updated.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2cc0b77a-df21-7007-5cec-4399d98c6dc6
Version Independent ID: efe5ae5e-ff64-f1c7-6920-4512fc8f36d1
Content: Azure Key Vault Configuration Provider in ASP.NET Core
Content Source: aspnetcore/security/key-vault-configuration.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@mkArtakMSFT  can you recommend the new resource for this topic now that @anurse doesn't own this?
@itsyashpal
Any chance you could PR this? At the top right of the article, select the Edit pen icon:

Select the pen icon again. Save.


github: explicitly set workflow permissions

current workflow only needs to read git content
if the workflow in the future does need write access (or even other read access than the git repo), it's good to
see permissions explicitly changing

For context: "pull_request" runs never have any write access anyway, so this
significantly changes only the "push" runs that happen when branches are
merged to develop.
Signed-off-by: Jussi Kukkonen<EMAIL_ADDRESS>Fixes #1662

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 97.442%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
3946

Relevant Lines:
4030

💛  - Coveralls

For anyone wondering: you can see the permissions by going to the job logs, opening "Set up job", then opening "GITHUB_TOKEN Permissions"

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+1.1%) to 98.548%

Totals

Change from base Build<PHONE_NUMBER>:
1.1%

Covered Lines:
3116

Relevant Lines:
3129

💛  - Coveralls


About the obtained results

Hi professor
I would first thank you for sharing such important code. I applied this code, however the obtained results on the KITTI are much lower than that published on the SOFT-VO paper. Could you professor explain the reason for that (may be some parts are missing in this code).
Regards

Hi achrafdjerida,
This code is not meant to recover the results published in the paper. Some parts are simplified if you look into the difference between paper and code. If you really want to get similar results as the paper, I would suggest you to improve the feature selection and matching part.

Thank you professor for the clarification . I will try to improve the steps you mentioned .


catalog-based m-test

the catalog-based m-test has issues when the ratio of forecasted events to observed events is large. @Serra314 can you upload the figures that you've made showing this behavior?

We discovered that the difference in the number of events between the simulated catalogues composing a forecasts and the number of observations may cause problems to the M-test.
The main reason behind is that we are estimating the score probability distribution calculating the score between the union of the simulated catalogues and each simulated catalogue. This means that if the simulated catalogues have an average of N events we are estimating the score probability distribution for a sample of length N. If the number of observations is different from N then the score between the union of the simulated catalogues and the observed events is coming from a different distribution than the one we have estimated using the forecasts. This leads to value of γ that, instead of being uniformly distributed, are very concentrated around 1 or 0 (depending on the fact that we are overestimating or underestimating the number of events).
One way to solve this is to estimate the score probability distribution using samples from the union of simulated catalogues instead of the forecasts in a bootstrap fashion. Sampling from the union of the forecasts a number of events equals to the number of observed, and calculating the score between each bootstrap sample and the union of the simulated catalogues, yelds a sample of score values under the null hypothesis that the forecast and the observation comes from the same distribution. The γ values obtained in this way are correct and this can be applied to many different scores.
Below an example in which the simulated magnitudes and the observed ones both come from a GR law with b-value equals 1. I have used an observed sample with 100 events, while each simulated catalogues has a number of events which comes from a Poisson distribution with mean 1000. I have calculated the gamma values considering 1000 different observed samples against the same forecast. They correctly look uniform in all cases.

Those are instead in cases where the forecasts come from a GR law with a different b-value, while the observations still come to a GR law with b-value equals 1. We can see how the distribution of %gamma; values departs from uniformity. The faster it departs from uniformity changing b-value the more sensitive the score is to incoherence in magnitude distribution.

I have made similar observations with the catalogue-based M-test, S-test, and PL-test. The way I think about it is that if the simulated catalogs over- or underestimate the number of events, their scores will, on average, be "more perfect" and "less perfect", respectively. (Perfect meaning being close to the overall distribution coming from all simulations.) So, I have seen two models that use the same spatial distribution pass the S-test in one case, but fail in another case, because they didn't forecast the same number of events. Same for M- and PL-test.
I think that this can even lead to an M-test being passed when it shouldn't pass, if the number of forecasted events is wrong in the right way.

Hi Leila,
Your observations are correct. It is exactly what happens and the problem is not limited to the M-test. I also  believe it is possible to find the "correct" number of forecasted event to make an incorrect magnitude forecast to pass the test ( I have not tried though).
I am currently writing a paper on this showing the problem and proposing the resampling of the union forecast as possible solution. In practice, instead of using the synthetic catalogs as they are we create the fake synthetic catalogues by sampling the union forecast as many time as observations. This solves the problem at least when the magnitude distribution is the same across different synthetic catalogues (which I believe is the most common case). I'll share a link when we are close to submitting it.


How to get two passport objects

Since passport returns an object (which is shared by any call to require('passport'), how would I go about getting two separate passport objects that can have separate configurations and initializations?
In other words something like:
Passport = require('passport);
pp1 = new Passport()
pp2 = new Passport()

You may want to consider using named strategies . . .


Improve fusion in VAE

This does break the producer -> insert_slice fusion in unet, so number of dispatches regresses.

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
View this failed invocation of the CLA check for more information.
For the most up to date status, view the checks section at the bottom of the pull request.

That was a commit from Jakob. I re-ran the check after you updated the base branch.


After installing, Brackets not opening in kali linux

Prerequisites

[ ] Can you reproduce the problem with Debug -> Reload Without Extensions?
[ ] Did you perform a cursory search to see if your bug or enhancement is already reported?
[ ] Did you read the Troubleshooting guide?

For more information on how to write a good bug report read here
For more information on how to contribute read here
Description
[Description of the bug or feature]
Steps to Reproduce

[First Step]
[Second Step]
[and so on...]

Expected behavior: [What you expected to happen]
Actual behavior: [What actually happened]
Versions
Please include the OS and what version of the OS you're running.
Please include the version of Brackets. You can find it under Help -> About Brackets (Windows and Linux) or Brackets -> About Brackets (macOS)

I too can't open Brackets in latest Kali

Same here

Same here. Please help!


postgresql password in quotes?

This line fails for me if the password is not in quotes.
https://github.com/seung-lab/CAVEdeployment/blob/9f3b106267f6146127bdf48864722684206a171d/infrastructure/global/launch_cluster.sh#L21

Only an issue if using special characters.

yeah agreed this should be changed..


Secret scanning: Add a custom help link to push protection blocks

Summary
GitHub Advanced Security customers can include a link for their developers to navigate to when GitHub detects secrets in their commits and blocks them from pushing.
Intended Outcome
Admins can provide additional support to developers via a link to internal tools.
How will it work?
When enabling push protection on an organization, admins can add an optional link to present to developers when they're blocked from pushing code.

🚢 This has shipped to dotcom: https://github.blog/changelog/2022-08-24-secret-scanning-admins-can-now-provide-a-link-to-display-when-a-push-is-blocked/. Leaving open to track for GHES release.

🚢 This has shipped with the GHES 3.7 release<EMAIL_ADDRESS>


Feature: SSL Settings

My company uses a proxy with SSL inspection, i cant find a way to set allowed trusted certificates.
Cheers

+1
I'm having the same issue.
npm's .npmrc and Atom apm's .apmrc files allow you to set a "strict-ssl" value to false. While I realize this is unsecure, it seems to be the only way to use these tools through our proxy.

Indeed, I've the same problem. How can I specify strict-ssl?

same problem here.

Starting Code with "NODE_TLS_REJECT_UNAUTHORIZED=0" seems to "fix" some issues and I actually get a list of extensions, but I still cannot install the extensions themselves. I get timeouts in the dev console after a while and that's it.

@maritz What do you mean by "Starting Code with"?
You put that into the settings?

Disclmaimer: This is not safe! (as is strict-ssl: false)
That's an environment variable. So in linux when starting vscode from command line you'd do this:
NODE_TLS_REJECT_UNAUTHORIZED=0 ./Code

Duplicate of #155
We will only support the http.proxyStrictSSL, not adding certificates to the trusted list.


Yatin: Watch ethereumtopaypal.com

Yatin requests the watch of the watch_keyword ethereumtopaypal\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

ethereumtopaypal\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

ethereumtopaypal\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ

Approved by tripleee in Charcoal HQ


2 second time window is to short

Is it possible to make the time window a little bit longer.  It never happens that I have enough time to react in 2 seconds. Transmitting is always starting in the next time window, 30 seconds later.
73 Ron
PD7RON

Thanks for your answer. It is now clear to me how I have to answer. I did it by holding my finger on the call and than chose call. That takes a to long time.
Now I understand that you have to swipe to the left. Than it is faster and it is working when the decode time is fast enough
73 Ron
PD7RON


Add Envoy::ExecutionContext.

Add Envoy::ExecutionContext, it can be inherited by subclasses to represent arbitrary information associated with the execution of a piece of code. Its activate/deactivate methods are called when the said execution starts/ends.
This is useful for things like resource accounting - see github issue for more details.
Commit Message: Add Envoy::ExecutionContext.
Additional Description:
Risk Level: Low.  It is no op in open source code.
Testing:  Unit test in test/common/common/execution_context_test.cc.
Docs Changes:  N/A
Release Notes:  N/A
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

Can you provide some context for this change? For example, is there a tracking issue or design doc that describes the goal of this work?

Can you provide some context for this change? For example, is there a tracking issue or design doc that describes the goal of this work?

Sure!  I'm working on project to break down cpu and memory usage of a (Google internal) Envoy server by some properties of a connection.  The tracking issue and design doc are also Google internal, but alyssawilk, RyanTheOptimist and abeyad all have access to.
It roughly works like this. Suppose each connection in my Envoy server has property X, and I want to know how much Envoy cpu and memory are used by different values of X, e.g. when X is 'priority', and potential values of 'priority' is one of ['HIGH', 'MEDIUM', 'LOW'], the question becomes how much Envoy cpu and memory is used by connections with 'HIGH', 'MEDIUM' or 'LOW' priorities. This can be addressed by doing a combination of two things:

When a envoy worker thread starts running code on behalf of a connection, set the connection's priority value in a thread-local variable.  When the thread finishes working on behalf of a connection, unset the thread-local variable.
Periodically sample the cpu(via SIGPROF) and memory(via TCMalloc sampling). Each time a sample is recorded, the sampling callback will read the thread-locals and declare the sampled cpu/memory is used by the corresponding property. e.g. In the SIGPROF handler, if the thread-local says priority=HIGH, the sampled cpu usage is used by high priority connections.  The sampled usage is then aggregated for human consumption.

(2) is purely Google internal implementation.  For (1), I found Envoy::ScopeTrackerScopeState already exists, all I need is to set the thread-local in its constructor and unset in destructor. I don't want to set/unset thread-locals directly in open source Envoy code, so I added Envoy::ExecutionContext and Envoy::ScopedExecutionContext such that in my internal implementation, I can set the thread-locals in Envoy::ExecutionContext::activate and unset in Envoy::ExecutionContext::deactivate.
(btw, the PR is not ready for review yet, I'll update the PR description once it's ready)

Removed "deps" since this PR no longer makes any deps changes.

Could you convert this to a draft PR while you work on it. That's a good signal to other maintainers that it's not ready to work on yet.

/assign @zuercher @RyanTheOptimist

/assign @penguingao

/assign @wbpcode
For cross-company review of a new feature.

Thanks for your contribution. It seems like a feature for debugging?
Anyway, please obey the style of Envoy code base. The method name should be lower camel case.
My another concern is the performance effect. This feature brings some additional hash lookup (and maybe repeatedly) and has no any gain for most of end users. And considering that this feature effect almost every connection/request, and has no a one simple bool switch, I would prefer guard it with a compile flag to disable it by default.
For example:
class ScopeTrackerScopeState {
public:
#ifdef ENVOY_ENABLE_EXECUTION_CONTEXT
  ScopeTrackerScopeState(const ScopeTrackedObject* object, Event::ScopeTracker& tracker)
      : scoped_execution_context_(object->scoped_execution_context()), registered_object_(object),
        tracker_(tracker) {
    tracker_.pushTrackedObject(registered_object_);
  }

private:
  ScopedExecutionContext scoped_execution_context_;

public:
#else
  ScopeTrackerScopeState(const ScopeTrackedObject* object, Event::ScopeTracker& tracker)
      : registered_object_(object), tracker_(tracker) {
    tracker_.pushTrackedObject(registered_object_);
  }
#endif

  ~ScopeTrackerScopeState() {
    // If ScopeTrackerScopeState is always used for managing tracked objects,
    // then the object popped off should be the object we registered.
    tracker_.popTrackedObject(registered_object_);
  }
  // Make this object stack-only, it doesn't make sense for it
  // to be on the heap since it's tracking a stack of active operations.
  void* operator new(std::size_t) = delete;

private:
  const ScopeTrackedObject* registered_object_;
  Event::ScopeTracker& tracker_;
};

Understand, I don't expect this to very costly, but I agree most users won't need it.
Instead of a macro, how about I add a runtime switch to enable it? Something like:
public:
ScopeTrackerScopeState(const ScopeTrackedObject* object, Event::ScopeTracker& tracker)
      : scoped_execution_context_(execution_context_enabled() ? object->scopedExecutionContext() : nullptr), registered_object_(object),
        tracker_(tracker) {

static void set_execution_context_enabled(bool enable) {
  execution_context_enabled() = enable;
}

private:
static std::atomic<bool>& execution_context_enabled() {
  static std::atomic<bool> enabled;
  return enabled;
}

This way I can call ScopeTrackerScopeState::set_execution_context_enabled(true) in my server.  Other users won't be affected.

/retest

Understand, I don't expect this to very costly, but I agree most users won't need it.

It's not expensive. Because the flat_hash_map is super fast is rarely become a bottleneck. However, the method will be called repeatedly at every filter chain hook and considering the more and more CPU overhead of request processing, I think be more strict to these change is not a bad thing.

Instead of a macro, how about I add a runtime switch to enable it? Something like:

I still prefer a compile flag. Esp considering that we already have some similar flags, like ENVOY_PERF_ANNOTATION, ENVOY_PERFETTO, etc.
You only need one additional line config to your .bazelrc to enable it.

If you insist on that the runtime flag is better and prefer it, I won't block it if other senior maintainer think it's OK. You can declare a new restart feature flag in the runtime_features.cc and use it to initialize a bool flag when it is used at first time.
static bool executionContextEnabled() {
  static const bool = Runtime::runtimeFeatureEnabled("envoy.restart_features.enable_execution_context")
  return enabled;

cc @envoyproxy/senior-maintainers cc @envoyproxy/runtime-guard-changes

If you insist on that the runtime flag is better and prefer it, I won't block it if other senior maintainer think it's OK. You can declare a new restart feature flag in the runtime_features.cc and use it to initialize a bool flag when it is used at first time.
static bool executionContextEnabled() {
  static const bool = Runtime::runtimeFeatureEnabled("envoy.restart_features.enable_execution_context")
  return enabled;

cc @envoyproxy/senior-maintainers cc @envoyproxy/runtime-guard-changes

Thanks, I added a runtime feature as suggested. This feature is NOT for debugging, so I want to avoid using macro to guard it.
@RyanTheOptimist : Are you ok with guarding this feature by a restart feature flag?

Yes, I think a restart flag is sufficient here. The cost of checking the flag should be negligible and avoiding #ifdefs makes the code a bit easier to test and reason about.

/retest

@wbpcode : Could you take another look at this? Thanks.

@wu-bin thanks for working on this - im wondering if this addition should have a changelog entry

Thanks for the comment, I added a changelog entry under the monitoring area.

thanks @wu-bin - i suggested adding as even tho there isnt user-facing config etc it seemed a significant change
it might be worth linking #32012
also - may be worth mentioning the runtime guard

Thanks, updated changelog as suggested.

Just a note for next time. Per the guidance from CONTRIBUTING.md, please do not reabase/force-push once a PR is out for review. Github handles that quite badly, alas. Instead you'll want to do git merge (which is less convenient, but ... sigh)

Chiming in belatedly, if you intend this to be optional long term it should have a proper config guard, ideally in bootstrap but optionally a non-reloadable/restart runtime feature.  Can you do a follow-up converting this over?

@wbpcode : Please see above for Alyssa's comment. I'm thinking we can just flip envoy_restart_features_enable_execution_context to true then delete it if there's no problems. WDYT? Are you still concerned about the runtime cost?

WDYT? Are you still concerned about the runtime cost?

Yeah, I will still prefer a global switch to disable it completely. This features make no sense for most users but bring actually overhead in the core data path to every user.
Although the overhead now is only the filter state searching, but accumulated minor overhead finally bring us big problem. See https://github.com/envoyproxy/envoy/issues/19103
This is why I initially prefer to use macro to control the feature.

WDYT? Are you still concerned about the runtime cost?

Yeah, I will still prefer a global switch to disable it completely. This features make no sense for most users but bring actually overhead in the core data path to every user.
Although the overhead now is only the filter state searching, but accumulated minor overhead finally bring us big problem. See #19103
This is why I initially prefer to use macro to control the feature.

Sure. It seems Envoy::Runtime::SnapshotImpl::getBoolean is also a hash lookup(similar to searching the filter state), I don't think we can just call it every time a scope object is needed. How about something like this:

Replace envoy_restart_features_enable_execution_context by a global boolean variable called execution_context_enabled.
In Envoy::Server::InstanceBase::initializeOrThrow, do  execution_context_enabled = runtime().snapshot().getBoolean("envoy.enable_execution_context", false).
Everywhere else, make sure executionContext returns nullptr when execution_context_enabled is false. Importantly, no filter state lookup is done when execution_context_enabled is false.

Thoughts?

In current implemention, Envoy::Runtime::SnapshotImpl::getBoolean will only be called once.

WDYT? Are you still concerned about the runtime cost?

Yeah, I will still prefer a global switch to disable it completely. This features make no sense for most users but bring actually overhead in the core data path to every user.
Although the overhead now is only the filter state searching, but accumulated minor overhead finally bring us big problem. See #19103
This is why I initially prefer to use macro to control the feature.

Sure. It seems Envoy::Runtime::SnapshotImpl::getBoolean is also a hash lookup(similar to searching the filter state), I don't think we can just call it every time a scope object is needed. How about something like this:

Replace envoy_restart_features_enable_execution_context by a global boolean variable called execution_context_enabled.
In Envoy::Server::InstanceBase::initializeOrThrow, do  execution_context_enabled = runtime().snapshot().getBoolean("envoy.enable_execution_context", false).
Everywhere else, make sure executionContext returns nullptr when execution_context_enabled is false. Importantly, no filter state lookup is done when execution_context_enabled is false.

Thoughts?

Sounds good to me. Cc @alyssawilk

WDYT? Are you still concerned about the runtime cost?

Yeah, I will still prefer a global switch to disable it completely. This features make no sense for most users but bring actually overhead in the core data path to every user.
Although the overhead now is only the filter state searching, but accumulated minor overhead finally bring us big problem. See #19103
This is why I initially prefer to use macro to control the feature.

Sure. It seems Envoy::Runtime::SnapshotImpl::getBoolean is also a hash lookup(similar to searching the filter state), I don't think we can just call it every time a scope object is needed. How about something like this:

Replace envoy_restart_features_enable_execution_context by a global boolean variable called execution_context_enabled.
In Envoy::Server::InstanceBase::initializeOrThrow, do  execution_context_enabled = runtime().snapshot().getBoolean("envoy.enable_execution_context", false).
Everywhere else, make sure executionContext returns nullptr when execution_context_enabled is false. Importantly, no filter state lookup is done when execution_context_enabled is false.

Thoughts?

Sounds good to me. But this essentially introduce a singleton, we may expect the non-reloadable/restart flag is server-dependent and cannot be singleton? (Not sure) Cc @alyssawilk

Thank you @wbpcode , I just sent you for this https://github.com/envoyproxy/envoy/pull/35722


Search: Javascript API

Public methods should be added so that DApps do not interact directly with bridge/indexing servers when making search queries.

Resolved by https://github.com/OriginProtocol/origin-js/pull/421 ✅


iOS/Android/Desktop app verifies token using web oidc client

Bug description
Owncloud desktop/mobile clients are verifying OIDC token using web oidc issuer provided in OCIS_OIDC_ISSUER environment variable. This bug prevents using some OIDC providers with token verification turned on (without PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD: "none")
Specs
OCIS Version: 3.0.0
Deployment type: docker
OIDC Provider: Authentik
Possible fix
Add possibility to define OIDC issuer for each oidc client i.e.
OCIS_OIDC_WEB_ISSUER="https://auth.provider.com/application/o/ocis-web/
OCIS_OIDC_DESKTOP_ISSUER="https://auth.provider.com/application/o/ocis-desktop/
OCIS_OIDC_IOS_ISSUER="https://auth.provider.com/application/o/ocis-ios/
OCIS_OIDC_ANDROID_ISSUER="https://auth.provider.com/application/o/ocis-android/

@legalgig I don't have a working authentik deployment at hand. Would you mind pasting some example access tokens issued for the various clients. I am somewhat surprised that the iss claim differs based on the client.
The current implementation of the access token verification was initially developed against keycloak and libregraph/lico.

Sure, here is one for ocis web interface (it was taken from the response)
{
  "iss": "https://auth.provider.com/application/o/ocis-web/",
  "sub": "4de6ff6d497cf4c1697a6557cfb0d115458c58b764dd7a43749bcd9a1d939547",
  "aud": "ocis-web",
  "exp":<PHONE_NUMBER>,
  "iat":<PHONE_NUMBER>,
  "auth_time":<PHONE_NUMBER>,
  "acr": "goauthentik.io/providers/oauth2/default",
  "email"<EMAIL_ADDRESS>  "email_verified": true,
  "name": "test",
  "given_name": "test",
  "preferred_username": "test",
  "nickname": "test",
  "groups": [
    "ocis-ios",
    "ocis-android",
    "ocis-web",
    "ocis-desktop"
  ],
  "azp": "ocis-web",
  "uid": "xzr4bW5FNa2XVLtsY9sfHz4WOOseVd7YHj3Cug3j"
}

And this one was taken from authentk interface because I'm not sure how to capture token for desktop application
{
  "iss": "https://auth.provider.com/application/o/ocis-desktop/",
  "sub": "de900b6e77b61c3a92ec730e59cbbde9b22bfd2180d6e74048ad3f65827da05b",
  "aud": "xdXOt13JKxym1B1QcEncf2XDkLAexMBFwiT9j6EfhhHFJhs2KM9jbjTmf8JBXE69",
  "exp":<PHONE_NUMBER>,
  "iat":<PHONE_NUMBER>,
  "auth_time":<PHONE_NUMBER>,
  "acr": "goauthentik.io/providers/oauth2/default",
  "amr": [
      "pwd"
  ],
  "email"<EMAIL_ADDRESS>  "email_verified": true,
  "name": "test",
  "given_name": "test",
  "preferred_username": "test-admin",
  "nickname": "test-admin",
  "groups": [
      "authentik Admins"
  ]
}

+1

Currently looking at the code that throws the error and was thinking whether or not a relatively simple OCIS_OIDC_ISSUER_SKIP_VALIDATION would be an option. It could just use the existing skipIssuerValidation field of the oidcClient struct. The required changes would require:

adding a simple environment/config var
changing:

-	if !claims.VerifyIssuer(issuer, true) {
+	if !c.skipIssuerValidation && !claims.VerifyIssuer(issuer, true) {

This would make the required change significantly smaller and less error prone than introducing multiple OIDC clients.

PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD: "none"

Are there any security concerns about setting this to none?

Are there any security concerns about setting this to none?

I don't think so. Even if we don't verify the token locally. It is still being validated as we use it for retrieving the userinfo from the IDP. Which cannot work with an invalid token.
.

I am similarly trying to setup ocis with Authentik. I have both Web and Android client. I can successfully login to the web client, however, trying to connect on the Android app gets me all the way through until I get an "Unsuccessful authorization" error. I have tried OCIS_OIDC_ISSUER_SKIP_VALIDATION to none, but it hasn't changed the result.

I think the fix as described in the issue would be the best approach to resolving this while still being able to maintain backwards compatibility. OCIS_OIDC_WEB_ISSUER, OCIS_OIDC_DESKTOP_ISSUER, OCIS_OIDC_IOS_ISSUER, OCIS_OIDC_ANDROID_ISSUER can just fallback to OCIS_OIDC_ISSUER if they are not set for their respective client.

@rhafer @butonic Do we need to do something here?

tl;dr no, close this issue. PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD=none does not reduce security.
Long version:
Authentik now supports the oidc webfinger endpoint: https://github.com/goauthentik/authentik/pull/10400
oCIS supports a rewrite of that endpoint because our clients currently try the oidc discovery at the ocis domain.
What I do not understand is why the different ocis clients lead to multiple providers/applications in Authentik?
👀
Ok after reading https://helgeklein.com/blog/owncloud-infinite-scale-with-openid-connect-authentication-for-home-networks/#authentik-configure-openid-connect-idp I now understand that OIDC clients are called providers in authentik and providers have a 1:1 relationship with applications.
AFAICT a single OpenID Connect Identity provider should support multiple clients. That is the whole idea of OpenID Connect: multiple clients being able to externalize the user and credential management to an identity provider.
And it seems to me that authentik already does support multiple clients as per the yaml config in https://helgeklein.com/blog/owncloud-infinite-scale-with-openid-connect-authentication-for-home-networks/#authelia-configure-openid-connect-idp
Setting PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD=none is perfectly fine. The proxy then queries the userinfo using the access token and can prevents access based on the response. PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD=jwt can be used with PROXY_OIDC_SKIP_USER_INFO=true to support ADFS.
Also, OIDC actually recomments the access token to be opaque to prevent leaking data that might find its way into a JWT. PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD=jwt may be used with PROXY_OIDC_SKIP_USER_INFO=true to save a trip to the IdP, which helps performance during login.
That being said, oCIS wants to support multiple identity providers in the future at least for migration purposes. However, we need to support assigning multiple identities to users, which is not a priority right now as it touches a lot of code.
So ... no, I don't think we should do something here. And I am against adding multiple env vars that influence the output of the wellknown lookup. We would have to look at the user agent to change the issuer url in the /.well-known/openid-configuration endpoint. That is the wrong approach. The correct fix would be to let our clients implement openid-connect discovery. Which might put even more burden and work on the hoster because he has to ensure that clients can look up the IdP based on their email.
That clients ask oCIS for the /.well-known/openid-configuration is IMO just a fallback when users try to login with a username and not an email address.


Add Travis CI for automated testing

Use Section 6 from my MOOC to guide your learning and implement TravisCI with a badge for this repo's README
https://www.udemy.com/reproducible-analytical-pipelines/

@myst3ria is it only admins who have permissions to link TravisCI with a Github repo within the ukgovdatasci organisation?

3rd party access for Travis CI has been granted, but just noticed there's already a Travis CI for Open Source application - @mammykins can you review and see if both are required


Ovsdbwork squashed

For everyone else, we are only planning to merge the work to the "bindings" branch of the repo. Once we support/test SSL, some HA testing, and more testing from everyone else and only after everyone else is comfortable, will we consider merging to master branch.

FIXED commit author


expression with filter and concatenation

I'm using following code to render gravatar images: JSFiddle

Expected result:
Actual result:

Workaround: JSFiddle

What if filters can't be placed at the end?
For example:
:suffix="totalPaymentsProcessed | currency + 'total processed'"

This is easily possible with un-bound attributes syntax:
suffix="{{totalPaymentsProcessed | currency}} total processed"

But sadly this is no longer allowed. Any workarounds that don't involve a new computed property or manually loading the filter into the component?


Use CMake default locations for wrappers

When CMAKE_INSTALL_PREFIX and CPACK_PACKAGING_INSTALL_PREFIX are both empty/unset, use the default for CMAKE_INSTALL_PREFIX as the installation prefix. This prevents the file(ABSOLUTE_PATH) statements from erroring out without an install prefix specified, and also alters the "correct" header file location in the generated wrapper. The relative paths are unaffected.

Unless there are any objections to this, I'll merge it in later today after I finish validating it against the math libraries.


How to add support for scripts from gui??

There is another program similar to this called automathemely which has support for scripts, including multiple scripts for light mode and dark mode however requires internet as only supports sunset sunrise, implementing such a feature has many advantages which are self explanatory

There is no such feature yet. Progress is tracked in #60


Bad conversion from non-UTC timestamptz with postgres

This comment here is incorrect:
https://github.com/launchbadge/sqlx/blob/2e1658e08b053f66102b5beabc7fdd4ac28e3a48/sqlx-core/src/postgres/types/chrono/datetime.rs#L79-L81
If you have a connection set to use a non-UTC timezone you will get a response with timestamps in that timezone. After overriding the executor to use PgValueFormat::Text for responses and adding some extra debugging code, running the following code:
let date = Utc.ymd(2020, 1, 1).and_hms(1, 1, 1);
let mut conn = pool.acquire().await?;
sqlx::query("SET TIME ZONE 'Europe/Berlin'")
    .fetch_optional(&mut conn).await?;
let row: (DateTime<Utc>,) = sqlx::query_as("SELECT $1::timestamptz")
    .bind(&date)
    .fetch_one(&mut conn).await?;

assert_eq!(row.0, date);

The assertion failed because the timezone on the response was not taken into account
[/tmp/tmp.ErEoBwofoQ/sqlx-core-0.4.0-beta.1/src/postgres/types/chrono/datetime.rs:79] ("decode", s) = (
    "decode", "2020-01-01 02:01:01+01",
)
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `2020-01-01T02:01:01Z`,
 right: `2020-01-01T01:01:01Z`', src/main.rs:19:5

I can't see any way to trigger this behaviour from outside sqlx-core since it always uses the binary format for responses, but it seems like a potential footgun to leave around in case this does someday become configurable.

This is possible to trigger if you use Executor::query() with a query string instead of sqlx::query(), et al (or the macros), as that uses the singular Query command which gets responses in text format instead of Parse/Bind/Execute.
We should be parsing DateTime using DateTime::parse_from_rfc_3339 which handles the timezone specifier correctly, not forwarding to NaiveDateTime.

As mentioned this doesn't occur normally because the binary result format is used. I managed to trigger it without editing sqlx-core to override that by using the direct text query as @abonander said:
    let row: (DateTime<Utc>,) = sqlx::query_as("SELECT '2020-01-01 01:01:01'::timestamptz")
                .fetch_one(&mut conn).await?;

But weirdly, if I also do this query before the SET TIME ZONE query, it causes it to not fail when queried afterwards.

@Nemo157 Thank you for your explanation. I totally misunderstood what this issue means. Sorry to trouble you.

I created a PR to fix this issue, but I'm not so confident about how to convert the output into types with timezone not determined on the application side, i.e. NaiveDateTime and DateTime<FixedOffset>.
If my understanding is correct:

For NaiveDateTime, the existing implementation implies that the timezone the output depends on differs according to the value of PgValueFormat.
For DateTime<FixedOffset>, the existing implementation implies that the output follows the local time of the server where the application is running.

In my PR, I kept these behaviors unchanged, but in my opinion, it may be natural that these values are determined according to the timezone of the database.

My PR https://github.com/launchbadge/sqlx/pull/1484 is waiting for reviewers to approve it
quoted:

1 workflow awaiting approval
First-time contributors need a maintainer to approve running workflows

@abonander Hi. I am currently looking at this problem and interested in reviving PR above or make a new one.
There is also problem that I cannot use custom timezones, because decode is implemented only for Utc, Local and FixedOffset. I think something like that would be more appropriate
impl<'r, Tz: Timezone> Decode<'r, Postgres> for DateTime<Tz> {
    fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
        ...
    }
}

I'll try to come up with solution if that's ok.

@Flowneee Happy to pair on this as I am also facing the same problem.
There is an associated problem that was fixed for SQLite here
Ideally the code below should be modified to use DateTime::parse_from_rfc3339(value) when the date is available as TEXT.
My expectations are, if I store a date: 2024-01-01T00:00:00-07:00, I expect the decoded value to be the same 2024-01-01T00:00:00-07:00 and NOT 2023-12-31T17:00:00-00:00
impl<'r> Decode<'r, Postgres> for DateTime<FixedOffset> {
    fn decode(value: PgValueRef<'r>) -> Result<Self, BoxDynError> {
        let naive = <NaiveDateTime as Decode<Postgres>>::decode(value)?;
        Ok(Utc.fix().from_utc_datetime(&naive))
    }
}

@abonander Can you provide guidance on whether we're on the right track?
Also related code that might need to be modified: if s.contains('+') feels flaky since it could also be '-'
PgValueFormat::Text => {
                let s = value.as_str()?;
                NaiveDateTime::parse_from_str(
                    s,
                    if s.contains('+') {
                        // Contains a time-zone specifier
                        // This is given for timestamptz for some reason
                        // Postgres already guarantees this to always be UTC
                        "%Y-%m-%d %H:%M:%S%.f%#z"
                    } else {
                        "%Y-%m-%d %H:%M:%S%.f"
                    },
                )?
            }

My expectations are, if I store a date: 2024-01-01T00:00:00-07:00, I expect the decoded value to be the same 2024-01-01T00:00:00-07:00 and NOT 2023-12-31T17:00:00-00:00

Note that Postgres does not store timezones. The timestamptz type is purely a convenience type to automatically convert input timestamps to UTC and output timestamps to the connection's timezone. (Personally I see no reason to use it in modern applications and think all timezone handling should be left to application side, but all the rust crates have decided that timestamp is naïve, not UTC).

All timezone-aware dates and times are stored internally in UTC. They are converted to local time in the zone specified by the TimeZone configuration parameter before being displayed to the client.
—https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-TIMEZONES

Try running SET TIME ZONE 'Europe/Berlin' before doing your query, you'll see it's not returned in -07 anymore.

@Nemo157 Thanks! Just double checked, sad since SQLite does store all of it as text so it can be queried back..

Also related code that might need to be modified: if s.contains('+') feels flaky since it could also be '-'

.contains('-') is always true because it's contained in %Y-%m-%d. I'm trying to support similar functionality in https://github.com/launchbadge/sqlx/pull/3511. You may take a look.


feat: fma-generic-contracts

Description
Introduces an fma-generic-contracts.md document in the security directory covering contract upgrades.
This was asked for by @mds1 as part of Stage 1.4 FMA review in https://github.com/ethereum-optimism/design-docs/pull/132.
These generic items were ported from notion.

This looks good to me given https://github.com/ethereum-optimism/design-docs/pull/140/files#r1820743365 is addressed by deleting that section and making sure that we have a generic contract upgrade failure mode, which I do believe that is already covered in this document with the other failure modes, cannot think of another generic contract upgrade failure mode unless i start to stretch like upgrade reverts/oog or something like that

FYI: I've updated https://github.com/ethereum-optimism/design-docs/pull/132 with the reference to DisputeGameFactory.setImplementation in fma-generic-hardforks.md 😄


Log File Path Should Be Immutable Once LogFile Starts

The LogFilePath property can currently be set after the LogFile has started.
This should be modified such that the property setter throw an Exception when the LogFile is running.

Fixed in commit 7d5ad2e12b5dc7ac935fb2d1fb00a1df3066001c


Look into the Babel transpiling issue in a section of populate hook.

A reminder for eddyystop.

What was the offending code again?

https://github.com/feathersjs/feathers-hooks-common/pull/84
https://github.com/feathersjs/feathers-hooks-common/pull/89

I think we can close this in favour of https://github.com/feathersjs/feathers/issues/608


Enable markdown in frontmatter texts (if needed)

See mdtext branch for how-to

This will ensure consistency among all text fields. Also, no need to use  or any html such as  on these fields.


[Bug]: Uncommitted changes doesn't come up for app settings & theme changes, also on comitting these changes progress bar doesn't come up

Is there an existing issue for this?

[X] I have searched the existing issues

Description
For a git connected app, when app settings or theme is modified, it doesn't reflect in bottom bar pull count. Also on committing these changes, progress bar doesn't come up
Steps To Reproduce

Connect an app to git
Add app settings or theme
Refresh the app and observe the count is not shown in bottom bar
Open git sync modal and commit changes
Observe progress bar doesnt come up

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
cloud

Now we can see the changes and also commit it


Update initial app load screen to show clearer search / register / log in actions

Description
LEARNER-3740

@salman2013 Ready for another pass.


what does n_head, d_model, d_k, d_v stands for?

n_head : split the embeddings into n_head parts to do multiple self-attention
d_model: embeddings vector size
d_k, d_v : key and values vector size, same as query size.