Stop outputting instructions as data bytes

RGBDS optimises instructions like LD [$FF40],a to LDH [$FF00+40],a, so these are encoded as data bytes using a macro to ensure exact reproduction of the original ROM (thanks to ISSOtm).
RGBDS automatically adds NOP instructions after STOP and HALT, so the disassembler will output these as data bytes if the instruction is not followed by a NOP in the original ROM.

RGBDS has flags to prevent this (-L for the former since 0.3.7, -h for the latter before 0.3.5); it should be possible to pass flags to mgbdis (possibly the same ones?) to leverage this behavior, and avoid data bytes in the middle of code, which might throw people off.

Ah yes, good suggestion.
I've now added a couple of options to support these: --disable-halt-nops and --disable-auto-ldh

Great!


Fix to avoid unintentional fails

Currently, rust-s3 reads /sys/hypervisor/uuid to determine whether the instance is EC2 or not. But some EC2 instances don't have /sys/hypervisor/uuid, but have /sys/class/dmi/id/board_vendor. This change will enable rust-s3 to run also in the latter type EC2 instances.
Reference:
https://github.com/scylladb/scylla/commit/2a4ba883c8e9c3351ff69064bb4d09954ff001c4#diff-23bb822fd3271daf086e185150f31517
Closes #67

This change is

@kenkoooo thank you for the PR, it will be on crates.io in a few minutes, version 0.18.9


Deploy to Azure button gives an error

I am receiving the following error when I click on the deploy to azure button:
"There was an error downloading the template from URI 'https://raw.githubusercontent.com/thivy/azure-chatgpt/main/infra/main.json'. Ensure that the template is publicly accessible and that the publisher has enabled CORS policy on the endpoint. To deploy this template, download the template manually and paste the contents in the 'Build your own template in the editor' option below. "

Same here


Packages v0.3.9 bump

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/approve


RFC 100: Public API for information on synchronization

RFC 100: Public API for information on synchronization
Authors: Nuno Preguiça, Gonçalo Cabrita, Jean Araújo
Status: Under discussion
(Under discussion / Rejected / Approved: assigned to X / Implemented: pull request X)
Context and Goal
Context
In AntidoteDB, updates are propagated asynchronously among DCs. A transaction accesses a snapshot of the stable updates known at the local DC (with stable meaning that atomicity of updates in a transaction and causal dependencies across transactions are satisfied).
Thus, when a transaction reads a value in a DC, it is possible some updates to that value have already been committed in some other DCs. Additionally, during the execution of a transaction, some other transaction may modify the read value in the same (or other) DC -- these updates are also not visible to the running transaction.
Goal
Provide an API to allow applications to have information about the up-to-date'ness of read data. This allows the application to reason about the staleness of read data, based on information available in the local DC.
We can define potential staleness as the time period for which the read data may be stale. This can be computed from the last synchronization with remote DCs, which establish the earliest time when an unseen update may have been issued (at some other DC).
Use Case
Synchronization status
A user-facing application may want to provide information to the user on the last time the local replica has been synchronized with the remote DCs.
Information on known changes
An application may want to prevent executing transactions on data that it knows has been modified concurrently.
API changes
External: AntidoteDB + protobuf

last_sync([bound_object()], TxId) -> {ok, [clock_time()]}.
For each object, it returns the earliest time it synchronized for the last time with some DC replicating the object -- let t1 be the last time it synchronized with DC 1, the result for object o is min(ti), with i the DCs where o is replicated.

last_sync_detail([bound_object()], TxId) -> {ok, [vectorclock()]}.
The same as before, but detailing the information of last synchronization with each DC.

known_stale([bound_object()], TxId) -> {ok, [boolean()]}.
For each data item, it returns whether the local DC has received an update that has not been reflected in the transaction snapshot.

Internal: transaction manager

last_sync([bound_object()], TxId) -> {ok, [clock_time()]}.
last_sync_detail([bound_object()], TxId) -> {ok, [vectorclock()]}.
known_stale([bound_object()], TxId) -> {ok, [boolean()]}.
As in the external API.

Internal: log

has_new_updates([bound_object(),vectorclock()]) -> {ok, [integer()]}.
Returns the number of known updates not reflects in the given vectorclock.

Desgin

last_sync([bound_object()], TxId) -> {ok, [clock_time()]}.

The return value can be computed from the snapshot time of the transaction. It is only necessary to process the operation in the component that records the snapshot time for the transaction.
API.last_sync([bound_object()], TxId) ->
	TM.last_sync([bound_object()], TxId).

TM.last_sync( list, TxId) ->
        snapshot_time = get_snapshot_time( TxId)
        return {ok, list.map(key ->
                            replicas = get_replica_set( key)
                            min( snapshot_time.filter( replicas)) ) }

NOTE: this implementation supports partial replication.

last_sync_detail([bound_object()], TxId) -> {ok, [vectorclock()]}.

The return value can be computed from the snapshot time of the transaction. It is only necessary to process the operation in the component that records the snapshot time for the transaction.
API.last_sync_detail([bound_object()], TxId) ->
	TM.last_sync_detail([bound_object()], TxId).

TM.last_sync_detail( list, TxId) ->
        snapshot_time = get_snapshot_time( TxId)
        return {ok, list.map(key ->
                            replicas = get_replica_set( key)
                            snapshot_time.filter( replicas) ) }

known_stale([bound_object()], TxId) -> {ok, [boolean()]}.

API.known_stale([bound_object()], TxId) ->
	TM.known_stale([bound_object()], TxId).

TM.known_stale([bound_object()], TxId) ->
        snapshot_time = get_snapshot_time( TxId)
        return {ok, list.map(key ->
                            LOG.has_new_updates([key,snapshot_time] > 0) ) }

LOG.has_new_updates([bound_object(),vectorclock()]) -> {ok, [integer()]}.

Return value based on the log information.
Implementation
Straightforward from the design.
Propagate calls to clocksi_readitem_server (the same way a read is done), where the snapshot time of the transaction is known.
Prior discussion
This feature has been discussed in the past in:

Syncfree deliverable D3.2 [https://syncfree.lip6.fr/attachments/article/46/d3.2.pdf]
Deepthi Devaki Akkoorath, Viktória Fördós, Annette Bieniusa: Observing the consistency of distributed systems. Erlang Workshop 2016: 54-55.

The last_sync information is very similar to the potential staleness that we are already calculating, right?
One issue that I see is that this information is divergent between different nodes. This means depending on where the respective clocksi_readitem_server is located, you might get different results. Are you planning to adapt the meta-data distribution to prevent this behaviour?
known_stale should be comparatively easy to compute; though, this will depend on the intraDC replication that we will establish.
@deepthidevaki What is your take?

last_sync if calculated from the snapshot_time, can be used to calculate the potential staleness. last_sync gives the exact time, while staleness is the difference between current time and what is being read(i.e. snapshot time). In this case, we don't need to propagate the call to clocksi_readitem_server, because transaction coordinator knows the snapshot time. Another way to get last_sync is to get it from the vector_clock of the partition which will be updated everytime when it receives a remote update. In this case, it does not reflect what is being read in the transaction, but what is available in the DC.
known_stale can be computed by materializer by checking the number of updates not included in the snapshot. Remote updates are immediately cached in the materializer, so it is up-to-date with what is in the log.

On Wed, Oct 4, 2017 at 5:23 AM, Annette Bieniusa<EMAIL_ADDRESS>wrote:

The last_sync information is very similar to the potential staleness that
we are already calculating, right?

Yes and no. You get similar information, but instead of computing the
difference from current time to the synchronization time, you return the
synchronization time.
I think this addresses your following issue.

One issue that I see is that this information is divergent between
different nodes. This means depending on where the respective
clocksi_readitem_server is located, you might get different results. Are
you planning to adapt the meta-data distribution to prevent this behaviour?

known_stale should be comparatively easy to compute; though, this will
depend on the intraDC replication that we will establish.

I would expect that you had to go to the log (or materializer to get this
information), while the previous one you could get immediately from the
snapshot time of the transaction.

@deepthidevaki https://github.com/deepthidevaki What is your take?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-334046398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8pDr2Yvk4TJc2YsKmSZ4izfOrPVRks5sowhbgaJpZM4PsWFb
.

On Wed, Oct 4, 2017 at 9:42 AM, Nuno Preguica<EMAIL_ADDRESS>wrote:

On Wed, Oct 4, 2017 at 5:23 AM, Annette Bieniusa<EMAIL_ADDRESS>
wrote:

The last_sync information is very similar to the potential staleness that
we are already calculating, right?

Yes and no. You get similar information, but instead of computing the
difference from current time to the synchronization time, you return the
synchronization time.

And the idea is to expose that by a public API  call.

I think this addresses your following issue.

One issue that I see is that this information is divergent between
different nodes. This means depending on where the respective
clocksi_readitem_server is located, you might get different results. Are
you planning to adapt the meta-data distribution to prevent this behaviour?

known_stale should be comparatively easy to compute; though, this will
depend on the intraDC replication that we will establish.

I would expect that you had to go to the log (or materializer to get this
information), while the previous one you could get immediately from the
snapshot time of the transaction.

@deepthidevaki https://github.com/deepthidevaki What is your take?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-334046398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8pDr2Yvk4TJc2YsKmSZ4izfOrPVRks5sowhbgaJpZM4PsWFb
.

Dr. Annette Bieniusa
University of Kaiserslautern
Department of Computer Science
AG Software Technology
Office: Building 32, Room 430
Phone: +49<PHONE_NUMBER>
Fax: + 49<PHONE_NUMBER>

On 4. Oct 2017, at 10:43, preguica<EMAIL_ADDRESS>wrote:
On Wed, Oct 4, 2017 at 9:42 AM, Nuno Preguica<EMAIL_ADDRESS>wrote:

On Wed, Oct 4, 2017 at 5:23 AM, Annette Bieniusa<EMAIL_ADDRESS>
wrote:

The last_sync information is very similar to the potential staleness that
we are already calculating, right?

Yes and no. You get similar information, but instead of computing the
difference from current time to the synchronization time, you return the
synchronization time.

And the idea is to expose that by a public API call.

Sure.

I think this addresses your following issue.

One issue that I see is that this information is divergent between
different nodes. This means depending on where the respective
clocksi_readitem_server is located, you might get different results. Are
you planning to adapt the meta-data distribution to prevent this behaviour?

I seem to be missing something… The point is that the synchronisation time might appear different (in the current design) for the vnodes.
This would need a fix to get always-advancing information.

known_stale should be comparatively easy to compute; though, this will
depend on the intraDC replication that we will establish.

I would expect that you had to go to the log (or materializer to get this
information), while the previous one you could get immediately from the
snapshot time of the transaction.

Yes, but what if the materialiser is replicated?

@deepthidevaki https://github.com/deepthidevaki What is your take?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-334046398,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8pDr2Yvk4TJc2YsKmSZ4izfOrPVRks5sowhbgaJpZM4PsWFb
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/SyncFree/antidote/issues/318#issuecomment-334089012, or mute the thread https://github.com/notifications/unsubscribe-auth/AAat8MTtnkKGkSXPlPhWxoe6-gmVDSl5ks5so0VBgaJpZM4PsWFb.

On Wed, Oct 4, 2017 at 9:36 AM, Deepthi Akkoorath<EMAIL_ADDRESS>wrote:

last_sync if calculated from the snapshot_time, can be used to
calculate the potential staleness. last_sync gives the exact time,
while staleness is the difference between current time and what is being
read(i.e. snapshot time).

Yes, that's the idea... we thought about having the staleness, as we have
discussed in the apst. The problem is that returning the time of sync
allows the application to compute the information locally.

In this case, we don't need to propagate the call to
clocksi_readitem_server, because transaction coordinator knows the snapshot
time.

OK... I was with Gonçalo trying to figure out here we needed to go to get
this info and it seemed it was to that server...

Another way to get last_sync is to get it from the vector_clock of
the partition which will be updated everytime when it receives a remote
update. In this case, it does not reflect what is being read in the
transaction, but what is available in the DC.

But here the idea is to get this for the transaction that is running,
which is accessing a snapshot taken at begin time.

known_stale can be computed by materializer by checking the number
of updates not included in the snapshot. Remote updates are immediately
cached in the materializer, so it is up-to-date with what is in the log.

It seemed to me that this would be only true if the materializer receives
all updates and this might not be the case for remote updates, but this is
an interesting question: what is the inter-relation between the
materializer and the inter-dc replication protocol?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-334087311,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8oSTGgwweyUn4Ycwn4Fhv0jcyLgyks5so0OYgaJpZM4PsWFb
.

It seemed to me that this would be only true if the materializer receives
all updates and this might not be the case for remote updates, but this is
an interesting question: what is the inter-relation between the
materializer and the inter-dc replication protocol?

Materializer receives all updates received by inter-dc protocol. Remote updates are cached in the materializer at the same time it is written to the log.

On Wed, Oct 4, 2017 at 12:49 PM, Deepthi Akkoorath<EMAIL_ADDRESS>
wrote:

It seemed to me that this would be only true if the materializer receives
all updates and this might not be the case for remote updates, but this is
an interesting question: what is the inter-relation between the
materializer and the inter-dc replication protocol?
Materializer receives all updates received by inter-dc protocol. Remote
updates are cached in the materializer at the same time it is written to
the log.
I guess in a future implementation of the materializer, this might not be
the case, right?
I suggest the following: route the request by the materializer. If the
materializer has the info, it replies. Otherwise, it will forward the
request to the log.
Does this make sense?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-334130696,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8ocJb4DoXOiyy_vQCCmlDi_-D5cDks5so3DWgaJpZM4PsWFb
.

Am 06.10.2017 um 15:14 schrieb preguica<EMAIL_ADDRESS>On Wed, Oct 4, 2017 at 12:49 PM, Deepthi Akkoorath<EMAIL_ADDRESS>
wrote:

It seemed to me that this would be only true if the materializer receives
all updates and this might not be the case for remote updates, but this is
an interesting question: what is the inter-relation between the
materializer and the inter-dc replication protocol?
Materializer receives all updates received by inter-dc protocol. Remote
updates are cached in the materializer at the same time it is written to
the log.
I guess in a future implementation of the materializer, this might not be
the case, right?
I suggest the following: route the request by the materializer. If the
materializer has the info, it replies. Otherwise, it will forward the
request to the log.
Does this make sense?

Did we clarify whether we have the materialiser replicated?

In the proposed design last_sync and last_sync_detail are calculated from the snapshot_time of the transaction irrespective of the list of bound_objects being passed. So bound_objects as parameters is unnecessary. Do you think, in a future implementation this will be needed, where last_sync is calculated differently for each keys irrespective of transaction?

The list of objects is there for supporting partial replication, where not
all DCs replicate all objects.
On Tue, Oct 10, 2017 at 1:45 PM, Deepthi Akkoorath<EMAIL_ADDRESS>
wrote:

In the proposed design last_sync and last_sync_detail are calculated from
the snapshot_time of the transaction irrespective of the list of
bound_objects being passed. So bound_objects as parameters is unnecessary.
Do you think, in a future implementation this will be needed, where
last_sync is calculated differently for each keys irrespective of
transaction?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/SyncFree/antidote/issues/318#issuecomment-335460512,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJZR8hly6EAvT5d3aPLiMbYxod-E8pj6ks5sq2cBgaJpZM4PsWFb
.


Organsise weapon data to be in structs

Description
The project will soon begin to bloat as we add more weapon data throughout development. Maintainability is becoming a problem as changes will need to be matching between each weapon attribute.
About the default values of the weapons:
The default values of the weapons represent the initial weapon attribute values of the weapon. These stats are not constant within each instant of the weapon and some might be modified with different stats before being Di 'ed into the weapon
Implementation

Create a struct called SWeaponAttributes

BP_PlayerCharacter:

Add two instances of the SWeaponAttribute struct. One being the default values of the weapon and the other being the active used values by the weapon.

Acceptance Criteria

[ ] The weapon now has structs for both default and active weapon attributes
[ ] Different starting values can be used as the default
[ ] Weapon should still be able to damage targets

Done in another ticket
Ticket is completed in #23 since it is needed to organise the data to be used for functions


GPU memory usage keeps increasing even hybridize with static_alloc when used in flask debug mode after mxnet 1.6.0post0.

Description
Hello, I'm using flask with mxnet to write a server. Since it is a web app, we want the GPU memory is fully static allocated.
However, as the title said, I found the GPU memory usage keeps increasing and then raise a OOM when the version of mxnet is 1.6.0post0 and 1.7.0, and if you are using mxnet 1.5.1, then all things are good. Since Flask debug mode uses multi-threading, I think it may be caused by some calls which are not thread-safe.

To Reproduce
This is a naive fLask server:
import mxnet as mx
import os
os.environ["MXNET_CUDNN_AUTOTUNE_DEFAULT"] = "0"
os.environ["MXNET_GPU_MEM_POOL_TYPE"] = "Round"

class Predictor(object):
    def __init__(self):
        ctx = mx.gpu(0)
        net = mx.gluon.model_zoo.vision.resnet50_v1()
        net.initialize()
        net.collect_params().reset_ctx(ctx)
        net.hybridize(active=True)
        max_h = 768
        max_w = 768
        _ = net(mx.nd.zeros(shape=(1, 3, max_h, max_w), ctx=ctx))
        self.ctx = ctx
        self.net = net

    def __call__(self, *args, **kwargs):
        max_h = 768
        max_w = 768
        x_h = np.random.randint(100, max_h)
        x_w = np.random.randint(100, max_w)
        xx = np.random.randn(1, 3, x_h, x_w)
        y = self.net(mx.nd.array(xx, ctx=self.ctx))
        return y.asnumpy().sum()

if __name__ == '__main__':
    import flask
    import tornado.wsgi
    import tornado.httpserver
    import os
    import cv2
    import numpy as np
    from flask_cors import CORS
    import os
    import cv2
    import json
    import logging
    import base64

    os.environ["MXNET_CUDNN_AUTOTUNE_DEFAULT"]="0"
    DEBUG = True
    PORT = 21500
    app = flask.Flask(__name__)
    CORS(app, supports_credentials=True)
    predictor = Predictor()

    @app.route('/test', methods=['POST'])
    def net_forward():
        try:
            r = predictor()
            return None
        except Exception as e:
            logging.exception(e)
            print("failed")
            return flask.jsonify(str(e)), 400

    print("starting webserver...")
    if DEBUG:
        app.run(debug=True, host='<IP_ADDRESS>', port=PORT)
    else:
        http_server = tornado.httpserver.HTTPServer(
            tornado.wsgi.WSGIContainer(app))
        http_server.listen(PORT, address="<IP_ADDRESS>")
        tornado.ioloop.IOLoop.instance().start()

And just run the following code to request the server:
import base64
import json
import time
import os
import numpy as np
import cv2

def remote_call(url):
    register_data = {"Pic": "xx"}
    data = json.dumps(register_data)
    import requests
    return requests.post(url, data)

if __name__ == '__main__':
    import glob
    import matplotlib.pyplot as plt
    while True:
        register_url = 'http://<IP_ADDRESS>:21500/test'
        while True:
            try:
                remote_call(register_url)
            except Exception as e:
                print(e)

Environment
I'm using flask 1.0.2 and tornado 5.1, but I think it is independent of the versions of flask and tornado.
We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:
curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python

paste outputs here
/data2/kohill/anaconda3/bin/python /data2/kohill/mx-detection/diagnose.py
----------Python Info----------
Version      : 3.7.0
Compiler     : GCC 7.2.0
Build        : ('default', 'Jun 28 2018 13:15:42')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 20.2.2
Directory    : /data2/kohill/anaconda3/lib/python3.7/site-packages/pip
----------MXNet Info-----------
Version      : 1.7.0
Directory    : /data2/kohill/anaconda3/lib/python3.7/site-packages/mxnet
Commit Hash   : 64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
64f737cdd59fe88d2c5b479f25d011c5156b6a8a
Library      : ['/data2/kohill/anaconda3/lib/python3.7/site-packages/mxnet/libmxnet.so']
Build features:
? CUDA
? CUDNN
? NCCL
? CUDA_RTC
? TENSORRT
? CPU_SSE
? CPU_SSE2
? CPU_SSE3
? CPU_SSE4_1
? CPU_SSE4_2
? CPU_SSE4A
? CPU_AVX
? CPU_AVX2
? OPENMP
? SSE
? F16C
? JEMALLOC
? BLAS_OPEN
? BLAS_ATLAS
? BLAS_MKL
? BLAS_APPLE
? LAPACK
? MKLDNN
? OPENCV
? CAFFE
? PROFILER
? DIST_KVSTORE
? CXX14
? INT64_TENSOR_SIZE
? SIGNAL_HANDLER
? DEBUG
? TVM_OP
----------System Info----------
Platform     : Linux-4.15.0-117-generic-x86_64-with-debian-stretch-sid
system       : Linux
node         : ubuntu
release      : 4.15.0-117-generic
version      : #118~16.04.1-Ubuntu SMP Sat Sep 5 23:35:06 UTC 2020
----------Hardware Info----------
machine      : x86_64
processor    : x86_64
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                48
On-line CPU(s) list:   0-47
Thread(s) per core:    2
Core(s) per socket:    12
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Model name:            Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
Stepping:              2
CPU MHz:               1200.672
CPU max MHz:           3300.0000
CPU min MHz:           1200.0000
BogoMIPS:              5002.04
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              30720K
NUMA node0 CPU(s):     0-11,24-35
NUMA node1 CPU(s):     12-23,36-47
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0060 sec, LOAD: 1.4688 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.1272 sec, LOAD: 1.2150 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1045)>, DNS finished in 0.10556268692016602 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0053 sec, LOAD: 1.4548 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0048 sec, LOAD: 11.7945 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.005016326904296875 sec.
----------Environment----------

Hi @kohillyang , I think it is not related to MXNet.
When there is a new connection, the library flask will create a new python process to handle the connection, which creates a new copy of MXNet instance predictor.

@wkcn but even if flask has created a new process, the GPU memory should be freed once the process ends. And the predictor is created in the main function, which should only be called once and has only one predictor instance. On the other side, if the main process has initialized a CUDA environment, the mxnet in the subprocess will fail when inference because their CUDA file descriptor can not be shared between the main process and the sub-process.
BTW. , the pid of the process and the id of the predictor remain unchanged. I print them using the following codes:
        print(id(self))
        print(os.getpid())

PS: ctx.empty_cache() is also not thread-safe. If you called it in two threads, the program would crash in some cases.

@wkcn predictor is created by predictor = Predictor(), not r = predictor(), since its children function __calll__ is overided.

@leezu could this be related to the issue fixed by #18328 #18363?

@kohillyang so you are creating a new predictor in every HTTP call? Thus yes, a new Block is created in every HTTP call and due to https://github.com/apache/incubator-mxnet/pull/18328 the parameter of the Block won't be deallocated.
https://github.com/apache/incubator-mxnet/pull/18328/files only contains Python changes. Would you like to try applying the changes to your MXNet files and see if the memory leak goes away. Thank you

Why do you think I'm creating a new predictor in each call? Apparently there is only one instance for Predictor.

You're creating a predictor inside def net_forward(). I don't know how often that is called.

Nevermind, I didn't read your def net_forward() carefully enough

Thus this is unrelated to #18328


Add RO version of 'acts on population of' to GO

@balhoff @pgaudet @ukemi
Chris merged the PR for adding 'acts on population of' to RO.
What is the correct order of operations now for replacing the RO relation in GO?
Do we first need to re-import RO, then replace the GOREL version in logical defs, and then obsolete the GOREL term?
Please advise.  Thx.

@pgaudet @ukemi @balhoff
Do we need to now obsolete 'acts on population of' in gorel?
Note that we also still have the relation 'regulates o acts on population of' in gorel, but it could refer to the RO version of 'acts on population of', right?  This would be analogous to a gorel relation like 'regulates o occurs in'.

Also need to document how new relations /changes from GOREL to RO are implemented in Noctua and P2GO.


It doesn't work and there's no error being thrown in the console.

Here's my example code. I can see the joy stick but it's not working. Not even on my iPad. The code is included here.
https://glitch.com/edit/#!/toothsome-trip?path=museum.html:9:167

Just tried on desktop and iphone 11 and the joystick works. I was never able to test this on iPad or Android devices.


As a forgetful courier, I want to be reminded of the orders I have to deliver today

so that I would not miss any order out.

Possible duplicate with #8

Duplicate with #8


cinnamon.xviewer: 2.8.3 -> 3.0.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/xviewer/versions.
meta.description for cinnamon.xviewer is: "A generic image viewer from Linux Mint"
meta.homepage for cinnamon.xviewer is: "https://github.com/linuxmint/xviewer"
meta.changelog for cinnamon.xviewer is: ""
Updates performed

Version update

To inspect upstream changes

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2/bin/xviewer passed the binary check.

/nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2/bin/.xviewer-wrapped passed the binary check.

2 of 2 passed binary check by having a zero exit code.

2 of 2 passed binary check by having the new version present in output.

found 3.0.2 with grep in /nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2

found 3.0.2 in filename of file in /nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2

directory tree listing: https://gist.github.com/952e9845cc17011ad9810d6f88cb550e

du listing: https://gist.github.com/b2f40723d77006deb3cadca7e943ca43

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
cinnamon.xviewer

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A cinnamon.xviewer https://github.com/r-ryantm/nixpkgs/archive/e6e0adacca4e652d1ca48fa1a5e2a406b026faea.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2
ls -la /nix/store/b5hj3l8hrqqz1yc8d2zrkcwaayy74zhy-xviewer-3.0.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

cinnamon.xviewer

Maintainer pings
cc @tu-maurice for testing.

Result of nixpkgs-review pr 133926 at e6e0adac run on x86_64-linux 1

1 package built successfully:

cinnamon.xviewer

1 suggestion:

warning: unused-argument
Unused argument: file.
Near pkgs/desktops/cinnamon/xviewer/default.nix:6:3:
  |
6 | , file
  |   ^


Remove scrolling hack, fix blocked initial scroll.

Fixes #1627
Reverts #1119, No longer able to reproduce in firefox (or chrome)
The initial hack was added in 1a805d1f482c9c2d19ace7acabe40b8059988e82

omg that is so much better.  thank you.
:dolphin: :dolphin:


⚠ Bofrost.lu has degraded performance

In 2a23a26, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1730 ms

Resolved: Bofrost.lu performance has improved in 38b721f after 6 minutes.


these credentials dont match the records

Laravel Version: 5.4
Adldap2-Laravel Version: 3.0
PHP Version: 5.6
Description: My active directory is working perfect but when i tried to login it says These credential do not match the records.

Hi @anukwgl,
What do you mean by My active directory is working perfect? Are you successfully connected to your LDAP server?
I need more information to work off of, such as what your using to authenticate users by (such as username or email), as well as your configuration.

I am authenticating by username, everything seems ok but when I gave my credentials it says doesn't match the records. Here is my config files:
adldap.php
'connections' => [
    'default' => [

'auto_connect' => true,
'connection' => Adldap\Connections\Ldap::class,
'schema' => Adldap\Schemas\ActiveDirectory::class,
'connection_settings' => [
'account_prefix' => env('ADLDAP_ACCOUNT_PREFIX', ''),
'account_suffix' => env('ADLDAP_ACCOUNT_SUFFIX', ''),
'domain_controllers' => explode(' ', env('ADLDAP_CONTROLLERS', 'ipaddress'')),
'port' => env('ADLDAP_PORT', 389),
'timeout' => env('ADLDAP_TIMEOUT', 5),
'base_dn' => env('ADLDAP_BASEDN', 'dc=,dc='),
'admin_account_suffix' => env('ADLDAP_ADMIN_ACCOUNT_SUFFIX', ''),
'admin_username' => env('ADLDAP_ADMIN_USERNAME', 'username'),
'admin_password' => env('ADLDAP_ADMIN_PASSWORD', 'Password'),
'follow_referrals' => false,
'use_ssl' => false,
'use_tls' => false,

Do I have to change anything else in my loginController:
middleware('guest', ['except' => 'logout']);
    }

}

middleware('guest', ['except' => 'logout']);
    }

}

Do I have to change anything else in my loginController:
public function username()
{
    return 'username';
}

Can you also post your config/adldap_auth.php file?
This issue is almost always configuration.

Do I have to change anything else in my loginController:

No, but if you're using a username instead of email address, you need to modify more things, such as the login view and your users table migration. All of which are mentioned in the documentation...

'connection' => env('ADLDAP_CONNECTION', 'default'),
'provider' => Adldap\Laravel\Auth\DatabaseUserProvider::class,
'resolver' => Adldap\Laravel\Auth\Resolver::class,
'importer' => Adldap\Laravel\Auth\Importer::class,
'rules' => [
Adldap\Laravel\Validation\Rules\DenyTrashed::class,
'scopes' => [
Adldap\Laravel\Scopes\UpnScope::class,
],
'usernames' => [
'ldap' => 'samaccountname',
'eloquent' => 'username',
'login_fallback' => env('ADLDAP_LOGIN_FALLBACK', false),
'password_sync' => env('ADLDAP_PASSWORD_SYNC', true),
'windows_auth_attribute' => ['samaccountname' => 'AUTH_USER'],
'sync_attributes' => [
    'username' => 'samaccountname',
    'name' => 'cn',

],

yes I followed the documentation and changed accordingly

For the account_suffix option in your config/adldap.php file, can you enter your domain suffix and try authenticating:
'account_suffix' => env('ADLDAP_ACCOUNT_SUFFIX', '@corp.acme.org'),

this is my adlogn.php and am following accordingly
function adLogin($username , $password)
{
$adServer = "ldap://ip address";
$ldap = ldap_connect($adServer);
$ldaprdn = 'abc' . "\\" . $username;

ldap_set_option($ldap, LDAP_OPT_PROTOCOL_VERSION, 3);
ldap_set_option($ldap, LDAP_OPT_REFERRALS, 0);

$bind = @ldap_bind($ldap, $ldaprdn, $password);

if ($bind) {
    $filter="(sAMAccountName=$username)";
    $result = ldap_search($ldap,"dc=abc,dc=lmn",$filter);
    ldap_sort($ldap,$result,"sn");
    $info = ldap_get_entries($ldap, $result);
    for ($i = 0; $i < $info["count"]; $i++) {
        if ($info['count'] > 1)
            break;
        /*          echo "
You are accessing  ". $info[$i]["sn"][0] .", " . $info[$i]["givenname"][0] ."
 (" . $info[$i]["samaccountname"][0] .")
\n";
                    echo '
';
                    var_dump($info);
                    echo '
';
                    $userDn = $info[$i]["distinguishedname"][0];
        */
        error_log("returning true");
        return true;
    }
    @ldap_close($ldap);
} else {
    error_log("returning false");
  return false;
}

}

Help me i even tried with 'account_suffix' => env('ADLDAP_ACCOUNT_SUFFIX', '@corp.acme.org'), but it says credentials don't match record

You need to enter your actual domain suffix, not the one I've given you in the example.
Also what is this code you've shown above? Is this how you're trying to authenticate users? Or is it just an example?

adladp_auth.php
'connection' => env('ADLDAP_CONNECTION', 'default'),
'provider' => Adldap\Laravel\Auth\DatabaseUserProvider::class,
'resolver' => Adldap\Laravel\Auth\Resolver::class,
'importer' => Adldap\Laravel\Auth\Importer::class,
'rules' => [
Adldap\Laravel\Validation\Rules\DenyTrashed::class,
'scopes' => [
Adldap\Laravel\Scopes\UpnScope::class,
],
'usernames' => [
'ldap' => 'samaccountname',
'eloquent' => 'username',
'login_fallback' => env('ADLDAP_LOGIN_FALLBACK', false),
'password_sync' => env('ADLDAP_PASSWORD_SYNC', true),
'windows_auth_attribute' => ['samaccountname' => 'AUTH_USER'],
'sync_attributes' => [
'username' => 'samaccountname',
'name' => 'cn',

],

adldap.php
 [

        'default' => [

            /*
            |--------------------------------------------------------------------------
            | Auto Connect
            |--------------------------------------------------------------------------
            |
            | If auto connect is true, Adldap will try to automatically connect to
            | your LDAP server in your configuration. This allows you to assume
            | connectivity rather than having to connect manually
            | in your application.
            |
            | If this is set to false, you must connect manually before running
            | LDAP operations.
            |
            */

            'auto_connect' => true,

            /*
            |--------------------------------------------------------------------------
            | Connection
            |--------------------------------------------------------------------------
            |
            | The connection class to use to run raw LDAP operations on.
            |
            | Custom connection classes must implement:
            |  \Adldap\Connections\ConnectionInterface
            |
            */

            'connection' => Adldap\Connections\Ldap::class,

            /*
            |--------------------------------------------------------------------------
            | Schema
            |--------------------------------------------------------------------------
            |
            | The schema class to use for retrieving attributes and generating models.
            |
            | You can also set this option to `null` to use the default schema class.
            |
            | Custom schema classes must implement \Adldap\Schemas\SchemaInterface
            |
            */

            'schema' => Adldap\Schemas\ActiveDirectory::class,

            /*
            |--------------------------------------------------------------------------
            | Connection Settings
            |--------------------------------------------------------------------------
            |
            | This connection settings array is directly passed into the Adldap constructor.
            |
            | Feel free to add or remove settings you don't need.
            |
            */

            'connection_settings' => [

                /*
                |--------------------------------------------------------------------------
                | Account Prefix
                |--------------------------------------------------------------------------
                |
                | The account prefix option is the prefix of your user accounts in AD.
                |
                | This string is prepended to authenticating users usernames.
                |
                */

                'account_prefix' => env('ADLDAP_ACCOUNT_PREFIX', 'abc'),

                /*
                |--------------------------------------------------------------------------
                | Account Suffix
                |--------------------------------------------------------------------------
                |
                | The account suffix option is the suffix of your user accounts in AD.
                |
                | This string is appended to authenticating users usernames.
                |
                */

                'account_suffix' => env('ADLDAP_ACCOUNT_SUFFIX', '@abc.lmn'),

                /*
                |--------------------------------------------------------------------------
                | Domain Controllers
                |--------------------------------------------------------------------------
                |
                | The domain controllers option is an array of servers located on your
                | network that serve Active Directory. You can insert as many servers or
                | as little as you'd like depending on your forest (with the
                | minimum of one of course).
                |
                | These can be IP addresses of your server(s), or the host name.
                |
                */

                //'domain_controllers' => explode(' ', env('ADLDAP_CONTROLLERS', 'corp-dc1.corp.acme.org corp-dc2.corp.acme.org')),
                'domain_controllers' => explode(' ', env('ADLDAP_CONTROLLERS', 'ipaddress')),
                //'domain_controllers' => array("ipaddress"),

                /*
                |--------------------------------------------------------------------------
                | Port
                |--------------------------------------------------------------------------
                |
                | The port option is used for authenticating and binding to your AD server.
                |
                */

                'port' => env('ADLDAP_PORT', 389),

                /*
                |--------------------------------------------------------------------------
                | Timeout
                |--------------------------------------------------------------------------
                |
                | The timeout option allows you to configure the amount of time in
                | seconds that your application waits until a response
                | is received from your LDAP server.
                |
                */

                'timeout' => env('ADLDAP_TIMEOUT', 5),

                /*
                |--------------------------------------------------------------------------
                | Base Distinguished Name
                |--------------------------------------------------------------------------
                |
                | The base distinguished name is the base distinguished name you'd
                | like to perform query operations on. An example base DN would be:
                |
                |        dc=corp,dc=acme,dc=org
                |
                | A correct base DN is required for any query results to be returned.
                |
                */

            //'base_dn' => env('ADLDAP_BASEDN', 'dc=corp,dc=acme,dc=org'),
                'base_dn' => env('ADLDAP_BASEDN', 'dc=abc,dc=lmn'),

                /*
                |--------------------------------------------------------------------------
                | Administrator Account Suffix
                |--------------------------------------------------------------------------
                |
                | This option allows you to set a different account suffix for your
                | configured administrator account upon binding.
                |
                | If left empty, your `account_suffix` option will be used.
                |
                */

                'admin_account_suffix' => env('ADLDAP_ADMIN_ACCOUNT_SUFFIX', ''),

                /*
                |--------------------------------------------------------------------------
                | Administrator Username & Password
                |--------------------------------------------------------------------------
                |
                | When connecting to your AD server, a username and password is required
                | to be able to query and run operations on your server(s). You can
                | use any user account that has these permissions. This account
                | does not need to be a domain administrator unless you
                | require changing and resetting user passwords.
                |
                */

                'admin_username' => env('ADLDAP_ADMIN_USERNAME', '*******'),
                'admin_password' => env('ADLDAP_ADMIN_PASSWORD', '*******'),

                /*
                |--------------------------------------------------------------------------
                | Follow Referrals
                |--------------------------------------------------------------------------
                |
                | The follow referrals option is a boolean to tell active directory
                | to follow a referral to another server on your network if the
                | server queried knows the information your asking for exists,
                | but does not yet contain a copy of it locally.
                |
                | This option is defaulted to false.
                |
                */

                'follow_referrals' => false,

                /*
                |--------------------------------------------------------------------------
                | SSL & TLS
                |--------------------------------------------------------------------------
                |
                | If you need to be able to change user passwords on your server, then an
                | SSL or TLS connection is required. All other operations are allowed
                | on unsecured protocols.
                |
                | One of these options are definitely recommended if you
                | have the ability to connect to your server securely.
                |
                */

                'use_ssl' => false,
                'use_tls' => false,

            ],

        ],

    ],

];

How do I Check the error y this is not loggging in?

Can you help me everything is perfect but i could not connect.


[FALSE-POSITIVE] https://virtual.bancodebogota.co

Domains or links
Please list any domains and links listed here which you believe are a false positive.
https://virtual.bancodebogota.co/
More Information
How did you discover your web site or domain was listed here?
the second option
2. Incorrectly marked as Phishing on Phishtank or OpenPhish?
Have you requested removal from other sources?
there are sources mark as whitelist I'm attaching the search of virustotal: https://www.virustotal.com/gui/url/95d862d1d30ea3cce8905ff1a3e6e3e39bf2dadfaa7f72cc257a4610496d8f63/detection
Additional context
The website is from Banco de Bogota is legitimate and trust.
:exclamation:
We understand being listed on a Phishing Database like this can be frustrating and embarrassing for many web site owners. The first step is to remain calm. The second step is to rest assured one of our maintainers will address your issue as soon as possible. Please make sure you have provided as much information as possible to help speed up the process.
Send a Pull Request for faster removal
Users who understand github and creating Pull Requests can assist us with faster removals by sending a PR on the whitelist.me file

https://github.com/mitchellkrogza/Phishing.Database/blob/master/whitelist.me/whitelist.me
Please include the same above information to help speed up the whitelisting process.

See comment in #67 and maybe follow @mitchellkrogza commit, which closed #67

No longer listed by our database @CMONITOREO but still listed by others so you have work to do elsewhere than here. Closing.


[zh] Update cert-manager docs link

Hi! This is the Chinese version of #30432 (split into a separate PR following the example of #28758 which was a similarly simple change).
/language zh

/lgtm
/approve
English version merged.


Change vertical scrolling

Vertical scrolling is now handled by adjusting the position of
TimeGraph. This removes the need for CaptureViewElement to
explicitly need to know about scrolling in TimeGraph.
Bug: Part of b/176086740 to remove the dependency of TimeGraph from Track.

Hi Florian, thanks for this! :)
I have a question. Did you test what's happening if we set TimeGraph to start at y = 20, instead of 0? I will need this soon because I would like to have the timeline in the top.

Edit: I commented on the wrong comment...
So again: Right, I didn't think about this... this will require some more refactoring. We should talk about the next steps here before merging this, maybe I should do this differently then.

I've changed my mind about this after our discussion yesterday, and reverted some of the changes. We still have VerticalScrollingOffset in TimeGraph, but this now directly updates the position of its children, so noone besides TimeGraph needs to know about this.


patch2pr: 0.25.0 -> 0.26.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/bluekeyes/patch2pr/releases.
meta.description for patch2pr is: Create pull requests from patches without cloning the repository
meta.homepage for patch2pr is: https://github.com/bluekeyes/patch2pr
meta.changelog for patch2pr is: https://github.com/bluekeyes/patch2pr/releases/tag/v0.26.1
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.26.1 with grep in /nix/store/pbxi46hj671jy9wg2y5v300031f3rw9f-patch2pr-0.26.1
found 0.26.1 in filename of file in /nix/store/pbxi46hj671jy9wg2y5v300031f3rw9f-patch2pr-0.26.1

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

patch2pr

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/pbxi46hj671jy9wg2y5v300031f3rw9f-patch2pr-0.26.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A patch2pr https://github.com/r-ryantm/nixpkgs/archive/cb2d357291f6f446dfa4f9c8b48f0ca593dec0fd.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/cb2d357291f6f446dfa4f9c8b48f0ca593dec0fd#patch2pr

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/pbxi46hj671jy9wg2y5v300031f3rw9f-patch2pr-0.26.1
ls -la /nix/store/pbxi46hj671jy9wg2y5v300031f3rw9f-patch2pr-0.26.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

patch2pr

Maintainer pings
cc @katrinafyi for testing.

Result of nixpkgs-review pr 327537 run on aarch64-darwin 1

1 package built:

patch2pr


⚠ Processors has degraded performance

In d653931, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 132 ms

Resolved: Processors performance has improved in e45d205.


issues when building for ECP5

Hi Nathan,
Thanks a lot for your work.
Trying to do the build for versa-ecp5 I found the following problems in meta-hdl (using current master):

the checksum for the licenses (in LIC_FILES_CHKSUM) for litespi and litex-pythondata-software-picolibc recipes are incorrect. When the build fails it gives the value that should be in the recipes

The build for linux-on-litex-vexriscv fails when building bios.elf. The problem seems to be the toolchain and the fact that a linux toolchain (provided by meta-oe?) is used instead of a bare-metal toolchain. I don't know what would be the best way to fix this. As a test I prepended PATH with the toolchain mentioned in linux-on-litex-vexriscv repository (installed in my build PC) so that the wrappers generated in do_create_toolchain_wrappers() in litexnative.bbclass point to this toolchain, and the it stopped failing.

The name of the generated .bit and .svf files are hardcoded to "top.bit" and "top.svf" in linux-on-litex-vexriscv.bb but for this target the generated files are "versa_ecp5.bit" and "versa_ecp5.svf".

If there is any stable tag/commit I should be using instead of 'master' branch please let me know.
I would also like to know if anyone could tell me how much work it would take to make this build work for a Xilinx Nexys4DDR target, either with the Vivado toolchain or with the open source ones.
Thanks!

I have pushed some commits to sort out the LIC_FILES_CHKSUM and .bit/.svf deploying.
However I don't see any errors or failures building the linux-on-litex-vexriscv recipe. Can you provide the error you see and the associated build configuration when you run bitbake. The litex project recently changed to using picolibc, however there is an odd dependency with picolibc where it depends on a toolchain that is not baremetal (e.g. newlib or glibc) for standard headers.
As for getting the build working for the Xilinx Nexys4DDR, it would be a matter of creating a recipe similar to the existing linux-on-litex-vexriscv.bb and sourcing the Xilinx tools install within the do_compile. I had looked at enabling access to the Xilinx tools from within OE previously (https://github.com/nathanrossi/meta-hdl/commits/xilinx-tools) which might be of help.

I discovered an issue where linux-on-lttex-vexriscv would fail to compile correctly when using the poky distro. This is likely the issue mentioned in second point. The error seen is similar to below:
| tmp/work/versa_ecp5-poky-linux/linux-on-litex-vexriscv/0+gitAUTOINC+708b0cbf31-r0/recipe-sysroot-native/usr/bin/riscv32-poky-linux/../../libexec/riscv32-poky-linux/gcc/riscv32-poky-linux/11.2.0/ld: ../libc/libc.a(libc_ssp_chk_fail.c.o): in function `__chk_fail':
| tmp/work/versa_ecp5-poky-linux/linux-on-litex-vexriscv/0+gitAUTOINC+708b0cbf31-r0/git/build/lattice_versa_ecp5/software/libc/../../../../../recipe-sysroot-native/usr/lib/python3.10/site-packages/pythondata_software_picolibc/data/newlib/libc/ssp/chk_fail.c:13: undefined reference to `write'
| collect2: error: ld returned 1 exit status
| make: *** [tmp/work/versa_ecp5-poky-linux/linux-on-litex-vexriscv/0+gitAUTOINC+708b0cbf31-r0/recipe-sysroot-native/usr/lib/python3.10/site-packages/litex/soc/software/bios/Makefile:68: bios.elf] Error 1

I have resolved this in https://github.com/nathanrossi/meta-hdl/commit/98b86e735e1ca95d0088a023621b81e0aff0dddd.
I will close this issue, if you still see errors building please re-open/comment on this issue with the exact error you observe.


Docs states "at no additional charge" which is I believe incorrect in some cases

From the pricing FAQ (https://azure.microsoft.com/en-us/pricing/details/sql-database/single/):
Backup storage is associated with automated backups of your instance. Increasing your backup retention period increases the backup storage that’s consumed by your instance. There’s no additional charge for backup storage for up to 100 percent of your total provisioned server storage. Additional consumption of backup storage will be charged in GB per month. For example, if you have the database storage size of 100 GBs, you’ll get 100 GBs of backup at no additional cost. But if the backup is 110 GBs, you’ll pay for the additional 10 GBs.
We have a customer that with the default Premium retention from DTU Model (35 days) is getting charged because it has a large database. IMO the docs should clearly explain that the backups COULD have additional cost.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9492cedc-cea0-af9d-29a0-52cbca8c5e21
Version Independent ID: 969a8654-8f95-d5de-351d-73c8e02e5d3a
Content: Azure SQL Database automatic, geo-redundant backups
Content Source: articles/sql-database/sql-database-automated-backups.md
Service: sql-database
GitHub Login: @anosov1960
Microsoft Alias: sashan

@christianarg Thank you very much for the insightful feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

In addition, it would be helpful if there were more cost transparancy for SQL Database backups. If I need to account for the backup costs for a given month, where do I go?

When using DTU model, there is no additional backup charge regardless of the size. The FAQ applies to the vCore model.

@anosov1960 your'e wrong. Feel free to re read my comment: We have a customer that with the default Premium retention FROM DTU MODEL (35 days) is getting charged because it has a large database.
The charges started 1 July 2018 just as the FAQ states. The FAQ does clarify when it's only about vCore, and the backup FAQ applies both to DTU and vCore.
It's not an opinion. MY CUSTOMER IS GETTING BILLED FOR BACKUPS USING DTU.

@christianarg, can you pls tell me how the charge looks on the bill. At the moment, we don't even have the backup meters for deployed. So there is no technical way of billing for the PITR backups. Is it possible that the charge is about the long-term retention backups? Those are optional and have always been billed but during the preview they were billed via Azure Backup service. Now it is included in the SQL database bill. Feel free to reach me (sashan at microsoft.com).

@anosov1960  (Sasha) helped with is issue and he was right. The charge we were seing was the storage Add on, which went on GA about the same time so that's what confuseds. Thanks again Sasha


Feature Suggestion: Invert scrolling

Hi, would be nice to have the option of 'natural scrolling', basically inverted, so it acts like a phone/tablet.

Yes I already tried that but it didn't work.
I should say that I'm using a Wacom pen and tablet, I wanted to use the middle button but that didn't work either, so it only works when the pen is hovering above the tablet with the modifier (shift) key pressed.
It's working for me like that though, I just wanted to invert the scrolling.

Did you restart the application after setting speed to a negative value? Also, could you share the output of defaults read com.emreyolcu.DragScroll?

All good!
I didn't realise I had to kill and then restart the app for the settings to take hold...
Thank you, fastest feature update after a request ever!
{
button = 5;
speed = "-1";
}


How to use SIV

I could not find an example of using AES-SIV or ChaCha20-SIV. Could you provide such an example please?

I'm afraid libsodium doesn't have AES-GCM-SIV or ChaCha20-Poly1305-SIV. Please see this FAQ section.

I'm afraid libsodium doesn't have AES-GCM-SIV or ChaCha20-Poly1305-SIV. Please see this FAQ section.


[CT-1362] [Bug] Toggling values when snapshotting a table with a key violation

Is this a new bug in dbt-core?

[X] I believe this is a new bug in dbt-core
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
If I have two values for the same primary key in two different rows in a given table that is snapshotted, it toggles between the two whenever I run the snapshot. (Snowflake adapter)
Expected Behavior
When I have a primary key error in a table I am expecting to snapshot, I would expect at least a warning (if not a full failure) if there is a primary key violation in the base table.
Steps To Reproduce
DBT Core 1.0.1

Load test data:
create or replace table sandbox.public.table2(pk int,city string); insert into sandbox.public.table2 values(1,'New York');

Run Snapshotting code:

{% snapshot DBT_TEST_SNAP_ALT%}

{{
    config(
      unique_key='PK',
      strategy='check',
      check_cols = 'all',
      invalidate_hard_deletes=True
    )
}}

select * from SANDBOX.PUBLIC.TABLE2

{% endsnapshot %}

Run:
dbt snapshot --select DBT_TEST_SNAP_ALT

Insert a key violating record into the table:
insert into sandbox.public.table2 values(1,'NOLA');

Re-run snapshot

Re-run snapshot again

Re-run snapshot again
Resulting table

Relevant log output
No response
Environment
- OS: Windows 10 Pro 19043.2130
- Python:3.8
- dbt:1.0.1

Which database adapter are you using with dbt?
snowflake
Additional Context
No response

Hey @kjstultz - I would also expect Snowflake to raise an error on the merge statement, saying that the key is not unique!
That's the default behavior, but if you've set the ERROR_ON_NONDETERMINISTIC_MERGE parameter to False instead, the merge will proceed, yielding nondeterministic results instead: https://docs.snowflake.com/en/sql-reference/parameters.html#label-error-on-nondeterministic-merge
By any chance, do you know if you have that parameter set for your account / user?

Interesting, good point @jtcohen6.  Currently we have that value always set to "True".  After combing through the code, I think the root of it is actually because dbt snapshot doesn't use the base table as the "merging" table. It creates a staged table on top that it uses, and from examining the code submitted, the PK violation in the table breaks the creation of that table.
As I look through the code more, I do really think the solution would be checking for a PK violation before the merge statement is initiated.

@kjstultz Ah - so just to confirm, the snapshot query itself is returning duplicate values of the unique_key?

As I look through the code more, I do really think the solution would be checking for a PK violation before the merge statement is initiated.

This is a fair thought. Today, we say it's the full responsibility of the end user to guarantee the uniqueness of the unique_key in the snapshot query.
One way to guarantee this is by creating another model (e.g. ephemeral model), put a unique test on it, a snapshot after it, and use dbt build (which will skip the snapshot if the unique test fails).
Or: Should the snapshot materialization just execute its own "unique test," implicitly, after creating the staging table and before running the merge? That does seem more ergonomic than the pattern I described above. Is there any reason we should make this an optional configuration, rather than turning it on by default for everyone (one more query than strictly needed)?

Forgot about this one, b

@kjstultz Ah - so just to confirm, the snapshot query itself is returning duplicate values of the unique_key?

As I look through the code more, I do really think the solution would be checking for a PK violation before the merge statement is initiated.

This is a fair thought. Today, we say it's the full responsibility of the end user to guarantee the uniqueness of the unique_key in the snapshot query.
One way to guarantee this is by creating another model (e.g. ephemeral model), put a unique test on it, a snapshot after it, and use dbt build (which will skip the snapshot if the unique test fails).
Or: Should the snapshot materialization just execute its own "unique test," implicitly, after creating the staging table and before running the merge? That does seem more ergonomic than the pattern I described above. Is there any reason we should make this an optional configuration, rather than turning it on by default for everyone (one more query than strictly needed)?

Definitely the second one. I'm not a big software development person, but I'd argue adding a parameter to the snapshotting config that defaults to "Check PK violation" to "FALSE" would be fine. A lot of use cases won't need this, but I've worked at enough organizations that have enough exceptions that having this parameter would be really really helpful.

@kjstultz thanks again for creating this issue! Appreciate that you invested the time at Coalesce in NOLA to figure out how to reproduce this tricky situation.
Reading the discussion between you and @jtcohen6, it sounds like the feature discussed thus far would look like this:

Add a new snapshot config parameter validate_uniqueness (default to false)
When validate_uniqueness is true, execute a uniqueness test after creating the staging table and before running the merge

Although this proposed feature wouldn't fix a snapshot table that already has duplicates, it would prevent duplicates from being added in the first place! Feels like a win to me, especially with it being configurable for those that (somehow) know that their "unique" key is truly unique.

Perfect! I think that'd be a great solution.

What do you think of the overall feature proposal here @jtcohen6 ?
What do you think about the name validate_uniqueness in particular? Got any other suggestion for this config variable name?


[11.x] Add Ability To Expect Specific Validation Error Keys In Tests

Adds method expectValidationErrors to InteractsWithExceptionHandling trait. This allows for asserting that specific validation keys are failing, which is not possible through the more general $this->expectException(ValidationException::class), where a ValidationException of any kind will suffice
For instance, you could have a test that ensures whether an invalid some_key field triggers a ValidationException. However, due to a bug in your code, some_key is actually valid, but some_other_key triggers a ValidationException. Since you're only expecting the exception and not the specific key, this test appears to pass (and masks a bug in your code)
Code loosely based on #42155
A few examples
// Will fail ❌
$this->expectValidationErrors(function () {
    // Do nothing
}, ['some_key']);

// Will fail ❌ (Exception will bubble)
$this->expectValidationErrors(function () {
    throw new Exception;
}, ['some_key']);

// Will fail ❌
$this->expectValidationErrors(function () {
    throw ValidationException::withMessages([
        'some_key' => 'Some error message',
    ]);
}, ['some_other_key']);

// Will pass ✅
$this->expectValidationErrors(function () {
    throw ValidationException::withMessages([
        'some_key' => 'Some error message',
    ]);
}, ['some_key']);

// Will pass ✅
$this->expectValidationErrors(function () {
    throw ValidationException::withMessages([
        'some_key' => 'Some error message',
        'another_key' => 'Another error message',
    ]);
}, ['some_key', 'another_key']);

// Will pass ✅
$this->expectValidationErrors(function () {
    throw ValidationException::withMessages([
        'some_key' => 'Some error message',
        'another_key' => 'Another error message',
    ]);
}, ['another_key']);

Unsure why tests failed, seemed to work for me locally. Will look into this tomorrow

@taylorotwell assertValid and assertInvalid require a TestResponse which typically means performing an HTTP request in the test, like through $this->postJson( ... ) or similar
I would find this to be useful for more unit-style tests that aren't concerned with sending an entire request and want to test a small, specific piece of logic. However I understand the concern of avoiding adding duplicate functionality/bloat. Appreciate the consideration


iOS: Feeds list section preview artifacts stuck on the screen

See the screenshots below. I was able to get the section title into a weird state by playing with cutting off the preview right before it activates. It kinda looks like the preview view didn't get cleaned up properly and it was just left on the screen.
Here it is right after I was playing with timings...

Here it is after collapsing the smart feed section. The empty preview view is just hanging out on the screen.

I've tried to recreate this and haven't had any luck.  Was this a preview for an Account, Folder, or Web Feed?
Did you have animations slowed down by any chance?  I'm having a heck of a time interrupting the animation at all.

I actually saw this when I was trying to reproduce a different issue. The initial issue I saw was that the separators would show and hide every time I expanded / collapsed the cell. When I was doing that, I was clicking the expand / collapse button sorta fast. I haven't been able to reproduce that, and I also haven't been able to reproduce this either.
What you see in the first screenshot is the empty preview right on top of the cell that was being previewed (an account cell) - no slow animations on. I was just clicking and holding and either letting go or dragging away right before the preview is displayed.

I still haven't been able to recreate this.  Maybe the new cropping preview code prevents it from triggering the bug?  If we find a way to recreate this consistently, I'll reopen this issue and see if we can do something to resolve it.


NamedArgumentForDataProviderRector only changing the first data provider

Hey there,
I've ran NamedArgumentForDataProviderRector on my project and noticed it only refactors the first #[DataProvider] it encounters. On my project, it missed about a 1/3 of the deprecations.
Changing getDataProviderMethodName to return an array of method names :
    private function getDataProviderMethodNames(ClassMethod $classMethod) : array
    {
        $methodNames = [];
        $attributeClassName = DataProvider::class;
        foreach ($classMethod->attrGroups as $attributeGroup) {
            foreach ($attributeGroup->attrs as $attribute) {
                if (!$this->isName($attribute->name, $attributeClassName)) {
                    continue;
                }
                foreach ($attribute->args as $arg) {
                    if ($arg->value instanceof String_) {
                        $methodNames[] = $arg->value->value;
                    }
                }
            }
        }
        return $methodNames;
    }

and calling it like this :
            $dataProviderMethodNames = $this->getDataProviderMethodNames($classMethod);
            if ($dataProviderMethodNames === []) {
                continue;
            }
            foreach ($dataProviderMethodNames as $dataProviderMethodName) {
                // ... rest of the code untouched
            }

fixes the issue. /cc @marcelthole
gnutix

Great :+1:  Could you send the PR with fix? :)

Sorry, too much on my plate these days.

No worries :+1:
Closing as duplicate of similar issue to keep focus: https://github.com/rectorphp/rector-phpunit/issues/346#issuecomment-2198370779

Without this fix, the rule is only partially doing its job, and the "optional feature" triggers deprecations that are outputted by PHUnit. So please, don't close this issue, it's a separate problem from the other you've linked.

I missunderstood then. We'll need a failing demo link first.

I added a failing testcase and fixed that case. We didn't had the case with multiple dataproviders on a single test method. But it should be fixed now :)


HDDS-8847. Fix javadoc of BucketLayoutAwareOMKeyRequestFactory#createRequest

What changes were proposed in this pull request?
Modify the BucketLayoutAwareOMKeyRequestFactory#createRequest annotation
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8847

Thanks @ayushtkn for the review.


Getting SIGILL with RUSTFLAGS='-C target-cpu=native'

Here's the error message I got:
error: failed to run custom build command for `libc v0.2.98`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installMEiNJj/release/build/libc-d4f829098caa5901/build-script-build` (signal: 4, SIGILL: illegal instruction)

Works fine if RUSTFLAGS is empty. I'm on Ubuntu 20.04 LTS, x86_64 and rustc version is 1.47.0. I got similar issue with frawk before which was due to lack of avx support on my hardware - perhaps it is the same issue here as well?

Also, installing using deb file worked too. But, the name is default.hck-linux-amd64.deb in the release page and hck-linux-amd64.deb in the README.

That's super odd! I haven't run into the native flag issue before. I also build on Ubuntu 20.04 when doing dev on an AMD cpu.
None of the other install methods use the native flag at least, as you noted with the deb build.
The deb file name should be fixed in the v0.5.3 release as well 👍

I'm going to close this for now on the assumption that this won't be a common thing, but if it keeps popping up for users I'll adjust the docs.


Out of memory error using WD14 captioning on Windows with RTX 4090

I have a set of 14 images with width of 512px and variable height. At first I tried to run it on bmaltais-gui, but encountered this error:

Then i tried to use this repo directly and error is still there:

No other program uses gpu, so all VRAM consumption comes from this script, here's a graph of vram usage:

I ran it through bmaltais gui, with and without installed CuDNN 8.6 dlls, neither worked.


Install features in container

Install features when they are added in server.xml or pom.xml/build.gradle dependencies.

To use an Open Liberty kernel image without pre-installed features, specify FROM open-liberty:kernel-slim in the Dockerfile.  See https://github.com/OpenLiberty/ci.docker/issues/193

User should have RUN features.sh in their Dockerfile if they need to install features to the image.
On server.xml changes, we should install features before they are enabled.
If featureUtility does not support Gradle style caches, we should consider mounting the install jars and calling those from the plugin instead.
We may need to start the container before mounting the jars and installing features.  Possibly skip RUN features.sh if we want to improve performance.


[release-v3.23] Auto pick #6415: Remove defaulting for FloatingIPs field - use code default

Cherry pick of #6415 on release-v3.23.
#6415: Remove defaulting for FloatingIPs field - use code default
Original PR Body below
Description

Defaulting this in the CRD causes problems on upgrades, because it
triggers an autoamtic Typha update to a Felix that doesn't yet
understand the new field, which causes Felix to
cyclically restart.
I think a proper fix for this would be to update Felix so that it
ignores updates for fields that it doesn't recognize, but this is a
quicker option.
Related issues/PRs

Fixes https://github.com/projectcalico/calico/issues/6294
Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

Remove API-level defaulting for FloatingIPs field - use code default instead

Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

@caseydavenport Hey, any chance of this being backported to 3.22?


adding bprec

Brand-Product Relation Extraction Corpora in Polish

@lhoestq I think this is ready for review, I assume the errors (connection) are unrelated to the PR :)

merging since the CI is fixed on master


Research wis2box ITS solution

The ITS solution seems to do more or less this:

data (bufr, but possibly other formats as well) is somehow uploaded to object store
a message is sent onto a queue
wis2box-management component listens on queue

downloads data
splits into smaller parts
inserts into object store
sends a message on another queue

wis2box-management seems to be made with an idea that several of the external components can be changed, but due to code structure and lack of documentation, it seems to be a fairly large job to do in practice.
If some of the components or the architecture of e-soh is to be different from wis2box, it is questionable whether we should keep or try to extend wis2box-management.


Adding Key Vault Support

Fixes #28

Documented here
Zip file here (Link expires at the end of the month so only use for testing this PR)
Merge #34 before this PR or you'll get a merge conflict as this builds on that branch as a base

@piaudonn I rebased this on main so that it would integrate all the other merged changes into the build and also run the pytest on the PR


Allow multiple projects references the MUX and projects theirselves have reference relatationship

Fix #599
To support compact, I injected a to the project for framework package. It introduced problem in _ComputeAppxPackagePayload if more than two projects are references the microsoft.ui.xaml and the two projects have reference relationship too.
For example, we have two projects: C# xUnit project and C# project:
C# xUnit project reference MUX nuget, it copied Compact.xaml to obj and compiled it.
C# project reference MUX nuget, it copied Compact.xaml again.
C# xUnit project reference C# project.
When compile C# xUnit project, _ComputeAppxPackagePayload aggregate Compact.xaml from both project in target _ComputeAppxPackagePayload, and task RemoveDuplicates detects this conflicts.
I have another implementation which removed Page from the .target and replaced it with .pri.

Cancelled and be replaced with pull #610


fix two broken links.

migrating-from-dnx.md became from-dnx.md

@mairaw Thanks for spotting that.
I will look at the review site before I merge master into live.


Add Terraform Kubernetes module comparison blog post

This pull-request proposes to add a blog post. The blog post introduces a new approach how to maintain Kubernetes resources using Terraform that is based on Kustomize vs. another approach that is based on Helm.

Reviewing the preview deploy, I noticed I had incorrectly left the date set to May from copying the previous post's markdown file. I fixed this and amended my commit.

Nice post, @pst. I did a light copy-edit for spelling, grammar, and punctuation.

Thank you for the thorough review @rolfedh. I really appreciate the help! I implemented all your suggestions, rebased and pushed.

@kbarnard10 @sftim from my perspective this is now ready for your review to get published when you have time. Should I update the date to a date in the future, and if so what date?

I spotted two last typos, one I missed correcting from the first review and one in the newly added section (every resources -> every resource). I also bumped the date of the post to today.

Thanks for the review @sftim. May I ask what the product that is being promoted and that prevents this from being published is, Terraform or Kubestack? Because Kubestack is not a commercial product and under the same OSS license as K8s. And I have no affiliation with HashiCorp.
The article describes a better way how to install Nginx ingress on Kubernetes. Both projects in the K8s GitHub org. The issues described further are a common topic in the terraform-providers channel in the Kubernetes Slack because users face them. So I think there's a good chance the content is relevant to many in the K8s community.

Here's  an example of what makes this look like a release announcement that promotes a particular project (external to Kubernetes):

After months of work and thousands of lines of Go, HCL, and YAML, I am very excited to share the release of the new Kustomize-based Terraform modules for Kubernetes.

The CNCF does differentiate between open and closed source solutions with labelling, but in terms of whether we promote external software projects, the answer is almost always No regardless of whether the external project is fully, partly or not open source.
We might be able to accept a different article that was more about using tooling (whether open source or not) to deploy a cluster with add ons. That article could mention external projects like kubestack. The sort of article we're more open to accepting typically reviews the ecosystem and details some alternatives, whilst also having a focus on a particular tech: for example, only talking in depth about solutions that rely on Terraform.
Another potential way to frame it would be to talk about in-project code such as https://github.com/kubernetes/ingress-nginx; the article I have in mind would be suitable for the Kubernetes blog because I'm imagining it to be clearly be focused on Ingress NGINX and (eg) ways to deploy it, rather than announcing the release of the Kubestack tooling.

I thought the this-is-new introduction would make it more interesting to read. But I can certainly rewrite that part.
If you ignore the fact that the Kustomize based Terraform modules are new for now, the article compares the Terraform/Helm integration with the Terraform/Kustomize integration. It uses Nginx ingress to have a realistic set of Kubernetes resources and make the examples more relevant to users. And it shows the code and workflow of how to use both and what's different.
The combination of K8s, Helm, Kustomize and Nginx ingress and the fact that it improves a major shortcoming of the integration with Terraform is certainly relevant to many readers of the K8s blog.
Assuming I change the announcement character of the post, would you see it as a fit then?

Assuming I change the announcement character of the post, would you see it as a fit then?

The information offered is relevant to Kubernetes and the audience for its blog. I'd love to see this article reshaped and published rather than discarded.
We do have to be fairly strict about not allowing product announcements for out-of-project code / products / services - can you imagine how many there might be if they were fair game?

Absolutely, I appreciate the review and am happy to work with the reviewers to make sure it meets the requirements. I'll start working on the changes and update the PR once I have something to show.

I've rewritten the introduction, as discussed @sftim.
Instead of the announcement character, it now explains the current situation and why I work on this topic. The post then continues with the same, unchanged, step-by-step comparison between the Helm-based and the Kustomize-based modules.
Finally, I've bumped the date to upcoming Monday.
Let me know what you think about the post now. Thanks!

By comparing the modules, the higher level abstraction, instead of the providers this article shows a more DRY approach. I think that has value for the day-to-day work of teams. Comparing the lower level providers may be an interesting read too, but is a different post - and one I also already contributed last year. There haven't been enough changes to justify another article about that in my opinion.


Windows could not start the AWSXrayDaemon service on Local Computer

I get this error when trying to setup and start xray daemon on local Windows 7:
Error 1067: The process terminated unexpectedly
Logs :
2018-10-10T09:55:41+02:00 [Info] Initializing AWS X-Ray daemon 3.0.0
2018-10-10T09:55:41+02:00 [Info] Using buffer memory limit of 1309 MB
2018-10-10T09:55:41+02:00 [Info] 20944 segment buffers allocated
2018-10-10T09:55:41+02:00 [Error] Unable to retrieve the region from the EC2 instance RequestError: send request failed
caused by: Get http://<IP_ADDRESS>/latest/meta-data/placement/availability-zone: dial tcp <IP_ADDRESS>:80: connectex: A socket operation was attempted to an unreachable network.
2018-10-10T09:55:41+02:00 [Error] Cannot fetch region variable from config file, environment variables and ec2 metadata.
This is my modified cfg.yaml :
# Maximum buffer size in MB (minimum 3). Choose 0 to use 1% of host memory.
TotalBufferSizeMB: 0
# Maximum number of concurrent calls to AWS X-Ray to upload segment documents.
Concurrency: 8
# Send segments to AWS X-Ray service in a specific region
Region: "eu-west-1"
# Change the X-Ray service endpoint to which the daemon sends segment documents.
Endpoint: ""
Socket:
  # Change the address and port on which the daemon listens for UDP packets containing segment documents.
  UDPAddress: "<IP_ADDRESS>:2000"
  # Change the address and port on which the daemon listens for HTTP requests to proxy to AWS X-Ray.
  TCPAddress: "<IP_ADDRESS>:2000"
Logging:
  LogRotation: true
  # Change the log level, from most verbose to least: dev, debug, info, warn, error, prod (default).
  LogLevel: "prod"
  # Output logs to the specified file path.
  LogPath: ""
# Turn on local mode to skip EC2 instance metadata check.
LocalMode: true
# Amazon Resource Name (ARN) of the AWS resource running the daemon.
ResourceARN: ""
# Assume an IAM role to upload segments to a different account.
RoleARN: ""
# Disable TLS certificate verification.
NoVerifySSL: true
# Upload segments to AWS X-Ray through a proxy.
ProxyAddress: ""
# Daemon configuration file format version.
Version: 2

I also have set the env var AWS_DEFAULT_REGION

Hi @abivolmv ,
From the error log, the daemon is not able to read region value. The daemon searches for the region value at from this location.  Parsing logic : Link
From the error log, it looks like the configured cfg.yaml file is not read by the daemon.
Are you using executable for Windows service?
If yes, it is possible because of the user used to start the service. The daemon service looks for the file in the directory, where the service is running and not in the custom directory, where you are running the script.
Usually, the windows services run from C:\Windows\System32 location. You would need to place configured cfg.yaml file at this location and start the daemon service.
Alternatively, you can pass command line option -c with the absolute path to cfg.yaml file.
Note -c parameter in the below script.
 sc.exe create AWSXRayDaemon binPath= "$daemonPath -f $daemonLogPath -c $configPath"
Please let me know if you have questions.
Thanks,
Yogi

Hi @abivolmv ,
The reason for above error is, the AWSXRayDaemon service is not able to access AWS Credentials while sending the segment. This may be due to user permissions.
Please make sure, your AWSXRayDaemon service has right permissions to access ~/.aws folder.
Alternatively, you can set environment variable for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, the daemon will automatically pick credentials.
Thanks,
Yogi

Thanks for the hint : I added reading permission to LOCAL SERVICE user on ~/.aws and it is now running.

No , actually I was to happy to close the issue. After adding that permission the service is starting but when there is a batch to be sent the same error appears :
2018-10-12T11:52:30+02:00 [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-10-12T11:52:31+02:00 [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-10-12T11:52:31+02:00 [Warn] Delaying sending of additional batches by 0 seconds

Setting the env vars worked. But is it possible to do it without the env vars ?

I am getting the following which i snot exactly the same as the errors above.  Any idea why?  My credentials are at "%UserProfile%.aws".  My client application in .Net Core picks them up just fine.
2018-10-18T18:28:02-04:00 [Debug] Failed to send telemetry 30 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.

Hi @rbanerjiaws ,
The reason for above error possibly is aws credentials are not available to the X-Ray daemon service.
For .NET Core application, AWS Dotnet sdk picks the credentials from %UserProfile%.aws location.
However, for X-Ray daemon which runs as a service locally, requires aws credentials to be made available using above methods as discussed, if the service is not able to recognize the aws credentials automatically.

Can you try setting the environment variables for aws credentials as discussed above.
Telemetry records reports  daemon statistics to X-Ray service at regular interval. After step 1,  can you check from the daemon logs whether the daemon is able to send Segments and Telemetry records? (you can follow script mentioned in the thread)

Please let me know if you have more questions.
Thanks,
Yogi

It worked with setting environment variables.  Just a minor note, the system environment variable needs to be set not the user's.

I have followed everything on this thread but still getting below error:
Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Hi, arcban02
Looks like a pretty common case: https://github.com/aws/aws-sdk-go/issues/992
As per previous comment: The AWS CLI uses credentials at path > dir "%UserProfile%.aws". Refer : https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html
Can you post the daemon logs, I'd lake to take a deeper look.
Can you confirm you have the ~/.aws folder, with a credentials file?
Can you confirm the AWSXRayDaemon service has right permissions to access ~/.aws folder?
Can you confirms the directory path above is on the PATH environment variable?
Did you try setting the environment variables, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY?
Thanks,
Sandra

Hi Sandra,
Thanks for looking into it.
Here are my daemon logs:
2019-07-29T10:53:22-07:00 [Info] Initializing AWS X-Ray daemon 3.1.0
2019-07-29T10:53:22-07:00 [Debug] Listening on UDP <IP_ADDRESS>:2000
2019-07-29T10:53:22-07:00 [Info] Using buffer memory limit of 203 MB
2019-07-29T10:53:22-07:00 [Info] 3248 segment buffers allocated
2019-07-29T10:53:22-07:00 [Debug] Using proxy address:
2019-07-29T10:53:22-07:00 [Debug] Fetch region us-west-2 from commandline/config file
2019-07-29T10:53:22-07:00 [Info] STS Endpoint : https://sts.us-west-2.amazonaws.com
2019-07-29T10:53:42-07:00 [Info] Using region: us-west-2
2019-07-29T10:53:42-07:00 [Debug] ARN of the AWS resource running the daemon:
2019-07-29T10:53:42-07:00 [Debug] No Metadata set for telemetry records
2019-07-29T10:53:42-07:00 [Debug] Using Endpoint: https://xray.us-west-2.amazonaws.com
2019-07-29T10:53:42-07:00 [Debug] Telemetry initiated
2019-07-29T10:53:42-07:00 [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.us-west-2.amazonaws.com
2019-07-29T10:53:42-07:00 [Debug] Using Endpoint: https://xray.us-west-2.amazonaws.com
2019-07-29T10:53:42-07:00 [Debug] Batch size: 50
2019-07-29T10:53:42-07:00 [Info] Starting proxy http server on <IP_ADDRESS>:2000
2019-07-29T10:54:18-07:00 [Debug] Received request on HTTP Proxy server : /GetSamplingRules
2019-07-29T10:54:36-07:00 [Debug] processor: sending partial batch
2019-07-29T10:54:36-07:00 [Debug] processor: segment batch size: 1. capacity: 50
2019-07-29T10:54:38-07:00 [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2019-07-29T10:54:58-07:00 [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2019-07-29T10:54:58-07:00 [Warn] Delaying sending of additional batches by 0 seconds
2019-07-29T10:55:19-07:00 [Debug] Failed to send telemetry 1 record(s). Re-queue records. NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
I have the  ~/.aws folder with credentials file.
Directory path is on PATH environment variable
I have also set setting the environment variables, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
However I am not sure how to confirm the AWSXRayDaemon service has right permissions to access ~/.aws folder
Thanks.

Hi arcban02,
If AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set, it shouldn't need to read in the credentials file. That's essentially an override - I'm really curious why that didn't work.
Confirm you have the following snippet/formats correct (and are set correctly in the env variables)
[default]
aws_access_key_id = ******************** (20 characters, upper case)
aws_secret_access_key =  **************************************** (40 characters, uppercase and lowercase letters, symbols)
More on the credentials file format: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials

Yes I do.

Let me see what other information I can dig up.

I think the daemon starts fine . The problem starts when I run a sample program that list buckets in S3. I do a  Aws::AssumeRoleCredentials. I am expecting to see traces of that call captured by Xray in daemon log. That is when I see [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.

The root cause is still the same, as per the previous entry from user abivolmv:
2018-10-11T09:23:02+02:00 [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2018-10-11T09:23:03+02:00 [Debug] processor: sending partial batch
2018-10-11T09:23:03+02:00 [Debug] processor: segment batch size: 1. capacity: 50
2018-10-11T09:23:03+02:00 [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

I do not think it's related to the AssumeRole call. Does it work without the Assume role?

I think you are right.

@arcban02 did you set the system's or user's environment variables ? It must be set on system in order to work (don't ask me why). See @rbanerjiaws comment above.

Yes I have System environment variables set.

Is anyone looking into this ?

I was able to finally run the xray daemon process locally on windows, on my development machine.
I used "xray-process" package instead of "xray-service". I suppose the "process" is meant to be used as a one-time instance, ran manually, as opposed to "service" which is meant to be installed as a windows service.
https://s3.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-windows-process-3.x.zip
https://s3.dualstack.eu-west-1.amazonaws.com/aws-xray-assets.eu-west-1/xray-daemon/aws-xray-daemon-windows-service-3.x.zip
The "process" seems to work in a similar way. It also looks for a %USERPROFILE%/.aws folder and expects a config file there, similarly to what was mentioned earlier here in this thread.
I'm not using access keys. As a developer, I already have aws cli working fine, after aws sso login.
I wanted a one-off xray daemon instance for debugging, and I wanted it to use my current authentication session.
It turned out that the current xray daemon doesn't support how the profile/sso/session are organized in .aws, but it could be adjusted manually, and then it ran with no issues.
Here are all the details: https://github.com/aws/aws-xray-daemon/issues/200#issuecomment-1632311913