Segmentation fault on m1 mac

Trying simple example on m1 mac:
from ctransformers import AutoModelForCausalLM

llm = AutoModelForCausalLM.from_pretrained(
    "/path/to/starcoderbase-GGML/starcoderbase-ggml-q4_0.bin",
    model_type="starcoder",
    lib="basic",
)

print(llm("Hi"))

leads to segmentation fault. Model works fine with ggml example code.

Hi, ggml recently introduced a breaking change so existing models have to be re-quantized. This error happens when you are using a old model with the new ggml library. If you pull the latest changes from ggml repo or do a fresh clone, you should get the same error with example code as well.
Latest quantized models are available in this repo: https://huggingface.co/NeoDim/starcoderbase-GGML/tree/main If you have already downloaded from this repo, please check if they are the latest as they got updated just 1 day ago.
Please ensure you are using the latest version of this library:
pip install --upgrade ctransformers

and then run:
llm = AutoModelForCausalLM.from_pretrained(
    'NeoDim/starcoderbase-GGML',
    model_file='starcoderbase-ggml-q4_0.bin',
    model_type='starcoder',
)

print(llm('Hi', max_new_tokens=1))

Please let me know if this works. The reason I used max_new_tokens=1 is because currently it is slow on Mac M1 (https://github.com/marella/ctransformers/issues/5#issuecomment-1556263858). If this basic example is working, we can see how to improve the performance.

Latest quantized models are available in this repo: https://huggingface.co/NeoDim/starcoderbase-GGML/tree/main If you have already downloaded from this repo, please check if they are the latest as they got updated just 1 day ago.

I know, this is mine repo :)
It still crashes with segmentation fault

I know, this is mine repo :)

Oh, nice! :)
Can you please try building from source and let me know if it works:
git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
./scripts/build.sh

The compiled library will be located at build/lib/libctransformers.dylib which can be used as:
llm = AutoModelForCausalLM.from_pretrained(..., lib='/path/to/ctransformers/build/lib/libctransformers.dylib')

Compiled from source also crashes with segmentation fault.

Unfortunately it also segfaults

Were you getting the error while loading the model using from_pretrained() or while generating text using llm()?

Also were you getting the error while loading the model using from_pretrained() or while generating text using llm()?

While generating. Load is fine. Tokenizer also works

Thanks. Can you try running the following and let me know where it is throwing the error:
print('eval', llm.eval([123]))

print('sample', llm.sample())

Sample works fine. Eval leads to segmentation fault

% export PYTHONFAULTHANDLER=1
% python modules/test.py
Fetching 1 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 19239.93it/s]
Fetching 1 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2978.91it/s]
loaded
Fatal Python error: Segmentation fault

Thread 0x00000001711bf000 (most recent call first):
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/threading.py", line 324 in wait
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/threading.py", line 607 in wait
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/site-packages/tqdm/_monitor.py", line 60 in run
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/threading.py", line 1016 in _bootstrap_inner
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/threading.py", line 973 in _bootstrap

Current thread 0x00000001e4c2db40 (most recent call first):
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/site-packages/ctransformers/llm.py", line 241 in eval
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/site-packages/ctransformers/llm.py", line 320 in generate
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/site-packages/ctransformers/llm.py", line 362 in _stream
  File "/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/site-packages/ctransformers/llm.py", line 453 in __call__
  File "/Users/username/nn/text-generation-webui/modules/test.py", line 11 in <module>

Extension modules: charset_normalizer.md, yaml._yaml (total: 2)
zsh: segmentation fault  python modules/test.py
/opt/homebrew/anaconda3/envs/textgen/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

I'm pretty new in python world. I hope it can help to debug this issue.

With lldb I can see:
%  lldb `which python3.10`
error: module importing failed: Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'cpython_lldb'
(lldb) target create "/opt/homebrew/anaconda3/envs/textgen/bin/python3.10"
Current executable set to '/opt/homebrew/anaconda3/envs/textgen/bin/python3.10' (arm64).
(lldb) run modules/test.py
Process 60321 launched: '/opt/homebrew/anaconda3/envs/textgen/bin/python3.10' (arm64)
Fetching 1 files: 100% 1/1 [00:00<00:00, 19599.55it/s]
Fetching 1 files: 100% 1/1 [00:00<00:00, 3890.82it/s]
loaded
Process 60321 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x16edffff8)
    frame #0: 0x00000001897bade0 libsystem_pthread.dylib`___chkstk_darwin + 60
libsystem_pthread.dylib`:
->  0x1897bade0 <+60>: ldur   x11, [x11, #-0x8]
    0x1897bade4 <+64>: mov    x10, sp
    0x1897bade8 <+68>: cmp    x9, #0x1, lsl #12         ; =0x1000
    0x1897badec <+72>: b.lo   0x1897bae04               ; <+96>
(lldb) up
frame #1: 0x0000000105386810 libctransformers.dylib`ggml_graph_compute + 128
libctransformers.dylib`ggml_graph_compute:
->  0x105386810 <+128>: mov    x9, sp
    0x105386814 <+132>: sub    x23, x9, x8
    0x105386818 <+136>: mov    sp, x23
    0x10538681c <+140>: mov    x25, #0x0
(lldb) up
frame #2: 0x0000000105352960 libctransformers.dylib`gpt2_eval(gpt2_model const&, int, int, std::__1::vector<int, std::__1::allocator<int>> const&, std::__1::vector<float, std::__1::allocator<float>>&, unsigned long&) + 2252
libctransformers.dylib`gpt2_eval:
->  0x105352960 <+2252>: ldp    x24, x22, [sp, #0x20]
    0x105352964 <+2256>: ldp    x20, x8, [x24]
    0x105352968 <+2260>: sub    x8, x8, x20
    0x10535296c <+2264>: asr    x8, x8, #2
(lldb)

Thanks for the detailed info. It looks like you are using anaconda and in a different issue (https://github.com/tee-ar-ex/trx-python/issues/23#issuecomment-1113606685 not related to this library) someone pointed out that anaconda could be the cause.
So can you please try installing Python from https://www.python.org/downloads/ and see if it works. Once Python is installed, pip can be installed using python3 -m ensurepip --upgrade

I will try, thanks

Can you try building from source and see if it improves.

Sure. Now try using built from source

16 minutes starchat-alpha-q4_0 100% cpu no output with max_new_tokens=1 file test.py:
from ctransformers import AutoModelForCausalLM
from ctransformers import AutoConfig

config = AutoConfig.from_pretrained(
    "/Users/sergeykostyaev/nn/text-generation-webui/models/starchat-alpha-ggml-q4_0.bin",
    threads=8,
)

llm = AutoModelForCausalLM.from_pretrained(
    "/Users/sergeykostyaev/nn/text-generation-webui/models/starchat-alpha-ggml-q4_0.bin",
    model_type="starcoder",
    lib="/Users/sergeykostyaev/nn/ctransformers/build/lib/libctransformers.dylib",
    config=config,
)
print("loaded")
print(llm("Hi", max_new_tokens=1, threads=8))

Printed only "loaded"

I think there might be some issue in the library itself. Another user also reported same issue (https://github.com/marella/ctransformers/issues/1#issuecomment-1556250969) but I thought it was just running slow. Before the breaking changes in GGML, older version of this library was working on M1 mac but just very slow. Now it appears to be not working.
Can you also try running a LLaMA model which basically uses llama.cpp:
llm = AutoModelForCausalLM.from_pretrained(
    'TheBloke/LLaMa-7B-GGML',
    model_file='llama-7b.ggmlv3.q4_0.bin',
    model_type='llama',
)

I will try with llama.cpp model

Also inference on example code from library - https://github.com/ggerganov/ggml/tree/master/examples/starcoder run just fine.

I think there might be some issue in the library itself. Another user also reported same issue (#1 (comment)) but I thought it was just running slow. Before the breaking changes in GGML, older version of this library was working on M1 mac but just very slow. Now it appears to be not working.
Can you also try running a LLaMA model which basically uses llama.cpp:
llm = AutoModelForCausalLM.from_pretrained(
    'TheBloke/LLaMa-7B-GGML',
    model_file='llama-7b.ggmlv3.q4_0.bin',
    model_type='llama',
)

Looks like for now llamacpp models has the same issue on apple silicon

16 minutes starchat-alpha-q4_0 100% cpu no output with max_new_tokens=1 file test.py:
from ctransformers import AutoModelForCausalLM
from ctransformers import AutoConfig

config = AutoConfig.from_pretrained(
    "/Users/sergeykostyaev/nn/text-generation-webui/models/starchat-alpha-ggml-q4_0.bin",
    threads=8,
)

llm = AutoModelForCausalLM.from_pretrained(
    "/Users/sergeykostyaev/nn/text-generation-webui/models/starchat-alpha-ggml-q4_0.bin",
    model_type="starcoder",
    lib="/Users/sergeykostyaev/nn/ctransformers/build/lib/libctransformers.dylib",
    config=config,
)
print("loaded")
print(llm("Hi", max_new_tokens=1, threads=8))

Printed only "loaded"

45 minutes - nothing changes

Thanks for checking patiently. I will debug this later.
Can you please try one last thing: try installing an older version of this library and see if it works:
pip install ctransformers==0.1.2

llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')

print(llm('Hi', max_new_tokens=1))

Thanks for checking patiently. I will debug this later.
Can you please try one last thing: try installing an older version of this library and see if it works:
pip install ctransformers==0.1.2

llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')

print(llm('Hi', max_new_tokens=1))

Sure.

Thanks for checking patiently. I will debug this later.
Can you please try one last thing: try installing an older version of this library and see if it works:
pip install ctransformers==0.1.2

llm = AutoModelForCausalLM.from_pretrained('marella/gpt-2-ggml')

print(llm('Hi', max_new_tokens=1))

Looks like after downgrade issue still here.

Thanks. Tomorrow I will add a main.cc file to repo which can be run directly without Python. It should make it easy to debug the issue.

Hi, I added the main.cc file in debug git branch. Please check if it works:
git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
git checkout debug

./scripts/build.sh
./build/lib/main <model_type> <model_path> # ./build/lib/main gpt2 /path/to/ggml-model.bin

Also please send the output of both ./scripts/build.sh and ./build/lib/main commands.

Hi. Sure, will test it now.

%  ./build/lib/main starcoder ../text-generation-webui/models/starchat-alpha-ggml-q4_0.bin

model type : 'starcoder'
model path : '../text-generation-webui/models/starchat-alpha-ggml-q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
eval ... ✔
sample ... ✔
detokenize ... ✔
delete ... ✔

response : ''

%  ./scripts/build.sh
-- CTRANSFORMERS_INSTRUCTIONS: avx2
-- ARM detected
-- Accelerate framework found
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/sergeykostyaev/nn/ctransformers/build
[ 60%] Built target ctransformers
[ 80%] Building CXX object CMakeFiles/main.dir/main.cc.o
[100%] Linking CXX executable lib/main
ld: warning: directory not found for option '-L/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/'
[100%] Built target main

%  ./build/lib/main llama ../text-generation-webui/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin

model type : 'llama'
model path : '../text-generation-webui/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
eval ... ✔
sample ... ✔
detokenize ... ✔
delete ... ✔

response : '!'

%  ./build/lib/main starcoder ../text-generation-webui/models/starcoder-ggml-q4_0.bin

model type : 'starcoder'
model path : '../text-generation-webui/models/starcoder-ggml-q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
eval ... ✔
sample ... ✔
detokenize ... ✔
delete ... ✔

response : ''

%  ./scripts/build.sh
-- CTRANSFORMERS_INSTRUCTIONS: avx2
-- ARM detected
-- Accelerate framework found
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/sergeykostyaev/nn/ctransformers/build
[ 60%] Built target ctransformers
[ 80%] Building CXX object CMakeFiles/main.dir/main.cc.o
[100%] Linking CXX executable lib/main
ld: warning: directory not found for option '-L/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/'
[100%] Built target main

Thanks. Is this entire output of build script? It should print the line "Found Threads: TRUE"
Also not sure why ld: warning: directory not found for option '-L/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/' appears on a arm macos.

ld: warning: directory not found for option '-L/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/'
this is problem in my configuration. Shouldn't be a problem. And yes, this is entire output.

I'm suspecting the issue to be with threads library not being found because the errors you posted previously also show threads in error message.
When you build ggml repo, are you seeing a line which says Found Threads: TRUE?
Also can you please try removing the line set(THREADS_PREFER_PTHREAD_FLAG ON) in CMakeLists.txt and try building again and see if threads is appearing.

After removing line set(THREADS_PREFER_PTHREAD_FLAG ON) from file models/CMakeLists.txt:
%  ./scripts/build.sh
-- CTRANSFORMERS_INSTRUCTIONS: avx2
-- ARM detected
-- Accelerate framework found
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/sergeykostyaev/nn/ctransformers/build
[ 60%] Built target ctransformers
[100%] Built target main

%  ./build/lib/main llama ../text-generation-webui/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin

model type : 'llama'
model path : '../text-generation-webui/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
eval ... ✔
sample ... ✔
detokenize ... ✔
delete ... ✔

response : '!'

At least the LLaMA model is giving some output, so the C++ code is working. So the issue might be when loading the library into Python. I will search more about this and get back to you if I find a solution. Thanks for helping with the debugging.

Maybe this - https://stackoverflow.com/questions/54587052/cmake-on-mac-could-not-find-threads-missing-threads-found

Maybe this - https://stackoverflow.com/questions/54587052/cmake-on-mac-could-not-find-threads-missing-threads-found

I also saw this but cmake should fail with an error but it is successfully building. May be it found threads but simply not printing it. When you build ggml repo, are you seeing a line which says Found Threads: TRUE?

Maybe this - https://stackoverflow.com/questions/54587052/cmake-on-mac-could-not-find-threads-missing-threads-found

I also saw this but cmake should fail with an error but it is successfully building. May be it found threads but simply not printing it. When you build ggml repo, are you seeing a line which says Found Threads: TRUE?

No.

Maybe this - https://stackoverflow.com/questions/54587052/cmake-on-mac-could-not-find-threads-missing-threads-found

I also saw this but cmake should fail with an error but it is successfully building. May be it found threads but simply not printing it. When you build ggml repo, are you seeing a line which says Found Threads: TRUE?

No.

Thanks for checking. I think cmake is just not printing that it found threads library, otherwise it wouldn't work all.

Hi @marella, I've been mentioned in #1 and #5. I have been able to run quantized models for starcoder, starchat, llama, whisper and mpt so far. Nonetheless, none of them work in ctransformers:

Mac M1 64GB vRAM
Architecture: ARM

I get exactly the same error as @s-kostyaev, meaning the llm object keeps running forever without any change, using the models natively works just fine. We've been trying to use ctransformers and langchain but nothing works, any new information?
I have done everything mentioned in this repo as well, building from source doesn't work.

Hi @bgonzalezfractal, s-kostyaev was helping me debug the issue but I couldn't find the reason/solution to this yet. So far we found that:

Model is loading and tokenize is working but eval method is failing in Python
C++ code works fine on its own but it is not working when being called from Python

I will keep looking for a solution and will let you know on this thread if I find a solution or if I need your help in debugging the issue.
Can you also please run the following and share the output:
git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
git checkout debug

./scripts/build.sh
./build/lib/main <model_type> <model_path> # example: ./build/lib/main gpt2 /path/to/ggml-model.bin

Please share the output of ./scripts/build.sh and ./build/lib/main commands.

 %  ./scripts/build.sh
-- CTRANSFORMERS_INSTRUCTIONS: avx2
-- ARM detected
-- Accelerate framework found
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/sergeykostyaev/nn/ctransformers/build
[ 60%] Built target ctransformers
[ 80%] Building CXX object CMakeFiles/main.dir/main.cc.o
[100%] Linking CXX executable lib/main
ld: warning: directory not found for option '-L/usr/lib/gcc/x86_64-pc-linux-gnu/11.1.0/'
[100%] Built target main

%  ./build/lib/main llama ../LocalAI/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin

model type : 'llama'
model path : '../LocalAI/models/WizardLM-7B-uncensored.ggmlv3.q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
> [ 1 18567 ]
eval ... ✔
sample ... ✔
> 29892
detokenize ... ✔
> ','
delete ... ✔

Thanks @s-kostyaev, I was actually asking bgonzalezfractal to run it so that I can check and comapre the output on their system as well :)
Since you already built it, can you also run ./build/lib/main on a starcoder model, because yesterday it was giving an empty response.

Sure.
%  ./build/lib/main starcoder ../LocalAI/models/starchat-alpha-ggml-q4_0.bin

model type : 'starcoder'
model path : '../LocalAI/models/starchat-alpha-ggml-q4_0.bin'
prompt     : 'Hi'

load ... ✔
tokenize ... ✔
> [ 12575 ]
eval ... ✔
sample ... ✔
> 399
detokenize ... ✔
> ' A'
delete ... ✔

Thanks. So the C++ code works fine natively and doesn't have any issue. I will have to debug while it is failing from Python.

@s-kostyaev I found another issue https://github.com/LibRaw/LibRaw/issues/437#issue-1065648301 which looks similar to the error you posted previously https://github.com/marella/ctransformers/issues/8#issuecomment-1557635980
They mention it to be a stack size limit issue which gets worse with multiple threads.
So can you please try using threads=1 after building from source (I added some print statements):
git clone --recurse-submodules https://github.com/marella/ctransformers
cd ctransformers
git checkout debug
./scripts/build.sh

llm = AutoModelForCausalLM.from_pretrained(..., lib='/path/to/ctransformers/build/lib/libctransformers.dylib')

llm('Hi', max_new_tokens, threads=1)

Also please run with threads=4 and share both the outputs.
In above thread, they also suggested increasing stack size limit but I'm not sure what an ideal limit would be.

Sure. Will test it.

With single thread:
 %  python3 test.py

ggml_graph_compute: n_threads = 0
ggml_graph_compute: create thread pool
ggml_graph_compute: initialize tasks + work buffer
ggml_graph_compute: allocating work buffer for graph (26048 bytes)
ggml_graph_compute: compute nodes

And it stucked.

Are you using threads=1? because it is printing n_threads = 0!
Can you also please check with threads=4.

Thanks. I think I found the issue. I will make a new release and will let you know in sometime.

@marella sorry I've been working like crazy, I see @s-kostyaev executed the necessary commands, if you need anything else from my hardware just let me know, glad you guys found it.

No worries @bgonzalezfractal

@s-kostyaev I released a fix in the latest version 0.2.1 Please update:
pip install --upgrade ctransformers

and let me know if it works. Please don't set lib=... option.
Also please try running with different threads (1, 4, 8) and let me know if you see any change in performance.

Finally it works. Threads parameter works. It even works with conda now. Thank you!

Thanks a lot @s-kostyaev for helping in debugging the issue.


Read config keys env vars

Read Fawry config keys env vars

@m7mdkamal I agree but it would break backward compatibility as code will need to be changed to use the new param name


RFC : jenkins_plugin : puppet module to configure jenkins plugins using groovy

Hi all,
https://github.com/witjoh/puppet-jenkins_plugin.git
This is still work in progress.  Feel free to peek around and give your comments/suggestions.
In short this module does :

installs plugins and there dependencies
configure plugins using puppet classes/defines.
Mainly uses the exec resource and groovy code.

Grts

@ekohl if still interested, intial attempt is on github.

I saw it, but currently very busy so haven't had time yet to take a look.


Exception instance for OAuth2Error

Such noise!

Although I really, really don't like having an Exception instance on this.

It's a temporary workaround whilst re-factoring a far more egregious mess. There's a trello ticket to clean this up after.


Is this wrong

pub async fn find_upstream(&self, addr: SocketAddr, hbone_port: u16) -> Option<Upstream> {
        let _ = self.fetch_workload(&addr.ip()).await;
        let wi = self.info.lock().unwrap();
        wi.find_upstream(addr, hbone_port)
    }

The addr is upstream，maybe clusterip. So confused how fetch workload by it?

pub async fn find_upstream(&self, addr: SocketAddr, hbone_port: u16) -> Option<Upstream> {
        let _ = self.fetch_workload(&addr.ip()).await;
        let wi = self.info.lock().unwrap();
        wi.find_upstream(addr, hbone_port)
    }

The addr is upstream，maybe clusterip. So confused how fetch workload by it?

There is vip -> workloads managed by ztunnel  via xds. So in find_upstream, it will first lookup addr as clusterip if not, fallthough to lookup as workload_ip.
fn find_upstream(&self, addr: SocketAddr, hbone_port: u16) -> Option {
if let Some(wl_vips) = self.vips.get(&addr) {
// Randomly pick an upstream
// TODO: do this more efficiently, and not just randomly
let (workload_ip, target_port) =
wl_vips.iter().choose(&mut rand::thread_rng()).unwrap();

I mean in         let _ = self.fetch_workload(&addr.ip()).await;
    pub async fn fetch_workload(&self, addr: &IpAddr) -> Option<Workload> {
        // Wait for it on-demand, *if* needed
        debug!("lookup workload for {addr}");
        match self.find_workload(addr) {
            None => {
                if let Some(demand) = &self.demand {
                    info!("workload not found, sending on-demand request for {addr}");
                    demand.demand(addr.to_string()).await.recv().await;
                    debug!("on demand ready: {addr}");
                    self.find_workload(addr)
                } else {
                    None
                }
            }
            wl @ Some(_) => wl,
        }
    }

    pub fn find_workload(&self, addr: &IpAddr) -> Option<Workload> {
        let wi = self.info.lock().unwrap();
        wi.find_workload(addr).cloned()
    }

Very confused, it find workload by WorkloadStore.workloads
IIUC the key of  WorkloadStore.workloads is the workload IP not clusterip

This is the find_workload implement
    fn find_workload(&self, addr: &IpAddr) -> Option<&Workload> {
        self.workloads.get(addr)
    }


ASP.NET 5 on Windows Server Nano

Tutorials -> ASP.NET 5 on Nano Server

i'd love any hint on this ASP.net 5 on nano or ASP.net 4.x
i've got a nano-server running but its not clear to me even if its running IIS - i don't see an option to add a web server...
our current deploy doesn't work even tho its using WinRM - the current script we have does a Get-WmiObject call and that doesn't work (ConDep) so thats out.

I'd love to be part of the ASP.NET 5 on Nano Server documentation early drafts review  :) Thanks guys

I'm working on the instructions now ... I'll have notes for @Rick-Anderson shortly ... later tonight???!!! It's very rough but working thus far. Fingers crossed for a successful outcome! I'll ping back soon.

It looks like there is a special version of the HttpPlatformHandler for IIS 10 that hasn't been released but that is required to run .NET 5 apps on Nano Server. I say so because when I apply the config seen in IIS 8.5 for the module to the applicationHost.config file of IIS 10 it crashes the AppPool of the default app. I've sent a message to the IIS Team asking them for clarification on this.

Progress Report
GOT IT! It's running now (thanks to a critical tips from @davidso). I have a solid set of notes/steps. The scenario I went with includes: an Azure Nano Server ARM VM, PowerShell ISE, a tidbit of PSEdit, and a network share using an Azure Files container.
I'll screenshot everything and provide a Word doc. I'll E-mail it to @Rick-Anderson. If you want me to continue thereafter, I'll happily do so; but if the official doc will be handled internally, I'm sure what I send over will be a huge help to you. I should have it done by noon on Thursday.
@jjmartin @jchable If you want a copy of my doc, let me know, and I'll arrange to post it where you cats can download it.
:sparkles: Nano Server :sparkles: ... small in size ... BIG IN HEART! :heart:

Congrats and thanks a lot for your good job @GuardRex ! Could you share a copy of it ?

@jchable Thanks and sure thing. I'll ping you back here with a link to it. I should have it by Thursday on noon PST.

:+1:

@danroth27 @Rick-Anderson @jchable @jjmartin
Update
:tada: Happy New Year! :tada:
I'm getting this online minutes before partying my rear-end off!
The doc has reached 1st draft stage. I'll leave this link up through the weekend: http://rexwestus.blob.core.windows.net/public/ASPNET5-on-Nano-Server.docx
Notes:

It is a 1st DRAFT. I wrote it today and only made a one-pass edit on it.
The Troubleshooting section that deals with attaching Event Viewer remotely is there (screenshots and all) ... but BROKEN. I'll get some feedback to sort that out later.
My philosophy for tutorials includes: (1) be clear, (2) use a lot of pics, (3) show a lot of code samples, (4) provide extra info and notes to help the reader understand what the heck is going on, and (5) provide adequate jumping-off links for more information. Those were the marks I was trying to hit.

Feedback ... comments, criticisms, remarks, death threats! ... all welcome.

@danroth27 @Rick-Anderson @jchable @jjmartin
Update
:tada: Happy New Year! :tada:
I'm getting this online minutes before partying my rear-end off!
The doc has reached 1st draft stage. I'll leave this link up through the weekend: http://rexwestus.blob.core.windows.net/public/ASPNET5-on-Nano-Server.docx
Notes:

It is a 1st DRAFT. I wrote it today and only made a one-pass edit on it.
The Troubleshooting section that deals with attaching Event Viewer remotely is there (screenshots and all) ... but BROKEN. I'll get some feedback to sort that out later.
My philosophy for tutorials includes: (1) be clear, (2) use a lot of pics, (3) show a lot of code samples, (4) provide extra info and notes to help the reader understand what the heck is going on, and (5) provide adequate jumping-off links for more information. Those were the marks I was trying to hit.

Feedback ... comments, criticisms, remarks, death threats! ... all welcome.

@GuardRex that's good and detailed; nice work :+1:

@GuardRex that's good and detailed; nice work :+1:

@benaadams Thanks ... that means a lot to me coming from you. It was fun to work up; and in spite of being a little rough around the edges (1st draft and all), I think it turned out ok.

@benaadams Thanks ... that means a lot to me coming from you. It was fun to work up; and in spite of being a little rough around the edges (1st draft and all), I think it turned out ok.

NOTE: Talking to @benaadams ... have become aware that Azure will reserve some names used in the tutorial. There are a few spots (where Azure account/VM names are used in PASTE commands) that need to be deleted from the doc. Read past those PASTE commands ... I'll update and provide a revision here soon.

NOTE: Talking to @benaadams ... have become aware that Azure will reserve some names used in the tutorial. There are a few spots (where Azure account/VM names are used in PASTE commands) that need to be deleted from the doc. Read past those PASTE commands ... I'll update and provide a revision here soon.

NOTE: @benaadams is providing important feedback now. There will be a set of minor (but important) revisions. Will provide updated doc link ... uh ... late New Year's Day ... uh ... yeah ... VERY late New Year's Day! lol :smile:

NOTE: @benaadams is providing important feedback now. There will be a set of minor (but important) revisions. Will provide updated doc link ... uh ... late New Year's Day ... uh ... yeah ... VERY late New Year's Day! lol :smile:

Works like a treat!

Works like a treat!

Updated
The doc has been updated based on conversation with @benaadams ... thanks again for his help ...
http://rexwestus.blob.core.windows.net/public/ASPNET5-on-Nano-Server.docx
Updates

Removed PASTE commands that contain Azure-specific naming, as those names cannot be guaranteed available to tutorial followers, e.g., "nanodemo"
Stronger message on setting Azure security group to 'None' on Azure VM config
Added note on requirement for running WinRM service locally if adding the VM's public IP to local Trusted Hosts fails.

Updated
The doc has been updated based on conversation with @benaadams ... thanks again for his help ...
http://rexwestus.blob.core.windows.net/public/ASPNET5-on-Nano-Server.docx
Updates

Removed PASTE commands that contain Azure-specific naming, as those names cannot be guaranteed available to tutorial followers, e.g., "nanodemo"
Stronger message on setting Azure security group to 'None' on Azure VM config
Added note on requirement for running WinRM service locally if adding the VM's public IP to local Trusted Hosts fails.

Lot's of love for Nano ... lot's of love! @davidso @benaadams ... Thanks guys!

Lot's of love for Nano ... lot's of love! @davidso @benaadams ... Thanks guys!

@benaadams
:tada: Licked IT! :tada:
Updated the section dealing with remotely attaching Event Viewer on a management server (e.g., a 2012/R2 box with a nice GUI) to the Nano Server to see its Event Logs. The Remote Event Log Management firewall rules must be enabled on the Nano if you want to go from this ... :frowning: ... to this ... :smile:
The doc has been updated.

@danroth27 @Rick-Anderson
This doc only covers insecure PowerShell access to the Nano. The doc should at least include the steps for 5986 secure access ... along the lines of Refaat Issa's http://blogs.technet.com/b/nanoserver/archive/2015/11/19/nano-server-iaas-image-in-the-azure-gallery.aspx
The question is should the secure approach be the only approach here? If so, it will make the troubleshooting section more complicated, too. It might be a little beyond the scope of the purpose of the doc, which is perhaps the least config required for getting things running.
I'll at least workup a section for it.

@danroth27 @Rick-Anderson @davidso @benaadams @moozzyk @Pinpoint @jchable @jjmartin  TL;DR ... but this is about the security issues surrounding Nano and PS remoting, so I'll take my time with it (@blowdart smiles! :smile:).
Security Update on PS Remoting Options & Nano Server
Option 1: Use an Azure 2012R2 management VM for the demo
As the doc currently goes, interaction with Nano is via insecure PS remoting on 5985. Not good out in the open, but I would think fine if working with it behind a firewall. For example, the doc could instruct to spin up a 2012R2 management VM first, RDC into that, then follow the doc and do everything PS session-related with the Nano from that VM. This would be simple and secure.
I don't think a Nano public IP address hit on 5985 from another machine in Azure gets outside of Azure, even if the management VM and the Nano are in different datacenters. Of course, someone in-the-know should confirm that.
This approach also has a great benefit: All of the troubleshooting bits in the doc for using Server Manager and the remote Event Log work well with this approach. Options 2 & 3 are not going to be able to make this claim.
If this approach is adopted, however, it is absolutely required to add a few steps to the deployment to put the Nano on a Network Security Group and lock it down to port 80 for public access (Internet) ... maybe throw in a 443 option in case the reader knows they want to play with TLS, too.
Option 2: Use the Azure Portal to create the Nano VM and set it up post-deployment (only using secure methods of access) for 5986 secure PowerShell remoting
I spent a fair amount of time today working up creating a client authentication management cert locally (New-SelfSignedCertificate), getting it into Cert:\LocalMachine\Root [$store.Add($mycert)], exporting it to the share (Export-PfxCertificate) to get it accessible to the Nano. On the Nano side, one should be able to run a script with Set-AzureRmVMCustomScriptExtension that will grab the cert and get it installed with a certoc.exe -ImportPFX ..., then delete it from the share. This never touches the Nano over 5985.
Only prob with all of this ... Nano is very, VERY stripped down. A lot of PS commands aren't available. winrm CLI isn't available, so I don't know how to create the 5986 WinRM listener (yet, anyway). You can't do it via Group Policy either AFAIK. It should be possible to do it manually with New-Item/Set-Item in WinRM config values for HTTPS with the cert thumbprint ... but I didn't get that far.
This approach could work out to allow using the Azure Portal to create the Nano with the only modifications to the doc being: (1) Run a handful of extra commands locally to get the cert and get it on the share, and (2) Put a script on the share to take care of the Nano-side and it trigger it with a simple PS command. Again, everything performed without ever using 5985.
However, getting the listener running is still an unknown ... I didn't get that far today. I'd like to hear back from you before I spend more time with this. Note that Option 1 above and Option 3 (Refaat's approach) below are safer, so there might be an instant desire to drop this approach entirely from contention.
Option 3: Use the Refaat approach and drop the use of the Azure Portal entirely (with some gotchas to watch out for)
http://blogs.technet.com/b/nanoserver/archive/2015/11/19/nano-server-iaas-image-in-the-azure-gallery.aspx

The script is not digitally signed, so I had to paste it into a new script and save that to run it.
It doesn't lowercase the location automatically. If you accidentally use any uppercase letters, it will create a mismatch between the VM DNS name and the common name of the certificate, so be sure to provide it like this, for example, -Location 'west us'.
There is a significant delay near the end of the deployment ... several minutes after you can see the VM is deployed in the Azure Portal, the script looks like it's hung. It's not hung. It just takes a long while before it reports back from Azure that the VM has been deployed with deployment details.
Nit: If you don't supply a -VMSize, it defaults to Standard_A2 ... hey! ... that's cool ... if your getting free Azure because you work for Microsoft! :smile: ... it should probably default to Standard_A0.
Downsides: You lose setting up the Nano in the Azure Portal. I kind'a liked that bit. It's great that you can script/template your way to ARM Nano VM's ... I'm sure we'll all be doing that plenty, but it's also cool that you can login, do a little point-and-click in the UI, and you're deployment is on the way. No fuss, no mess! A Network Security Group should be added. You don't get Server Manager and Event Viewer fun bits this way.

Option 4: Hybrid of Options 1 & 3
Use the Refaat approach to create the Nano but add a Network Security Group to lock down the public side (80+443+5986 only). Also, the doc could have the reader deploy a 2012R2 VM. They can sort of choose on the PS remoting for how to hit the Nano, but the management VM would allow for the Server Manager and Event Log demo pieces of the doc.
Preliminary Conclusion
My very preliminary gut feeling ... Option 1 ... I like it for three reasons ...

Option 1 involves the Azure Portal for the Nano setup, where I frequently enjoy working.
Security should be covered and no security best practices (that I've heard of anyway) would be violated with Option 1. Of Options 1-3, actually, I think Option 1 is the safest (but I want to hear from security gurus!). I like Refaat's approach, too; but as it stands, it doesn't add a Network Security Group. Even if it did, you'd have to open 5986 publicly.
Option 1 allows for a great demo of remote monitoring with Server Manager and getting a nice GUI Event Viewer safely hooked up to your Nano.

@moozzyk Asks about the "containers" version or content. I like the idea, too, and I was going to suggest a section could be added to this one or a new doc for it. https://github.com/aspnet/Hosting/issues/539#issuecomment-168789381

I'm working with the Nano folks to get this working on a local Hyper-V host. That would eliminate all the remoting concerns.

There is a video on Channel 9 that shows how to do this end-to-end. https://channel9.msdn.com/Series/Nano-Server-Team/Demo-Building-a-Nano-Server-VHDX-for-a-Hyper-V-VM
I have that working and will explore that approach.

A few things jump to mind ...

The lowest barrier to entry is probably the Azure image MS provides (for a tutorial I mean) ... not even a VHD image spun up in Azure. There's significantly more that has to be explained and that can go wrong with a generated VHD. It also takes lot longer to do: The Nano Server bits are in a folder with the full ISO, so it can take forever for that multi-gigabyte monster to download over a slow connection.
A Hypervisor-based approach might be better as an add-on section, since the thrust seems to be AFAIK to get .NET 5 on Nano ... it would take (potentially) a lot more explaining ... and again, more can go wrong ... and probably will. :smile:
Using an Azure 2012R2 VM for management (prob on the same subnet, prob with the Nano using an NSG) will be required anyway, even if a 100% Hypervisor approach is taken [but I'm not familiar (yet) with PS remoting through the Hypervisor itself (if I have that straight from Snover's comments), and I do like that idea and am looking forward to trying it out]. PS remoting over a locked-down subnet should be plenty secure and in-line with best practices. PS remoting has a lower barrier to entry.

I got pretty far with Hypervisor last night. I'm sitting down now to push ahead with it. What I'm doing is connecting an Azure 2012R2 with an installed Hypervisor (not via Roles & Features ... by PS command) to talk to an Azure Nano. I have the Hypervisor talking to the Nano ... I don't have the Nano playing nicely with it yet ... maybe I still need to open a few more ports. If they tell you all of the ports that need to be open on the Nano (and a full list of the required services would be nice, too), please ping me back.

I'm working with the Nano team. Looks like they have already added a cert (or will add one soon).  I agree, we'll stick with the Azure image route for now.

Cool.
In the vid, Refaat is not using any kind of special Hypervisor-based remoting. I thought that's what Snover was talking about in some vids ... that there is a Hypervisor-based PS remoting that didn't run through the PowerShell CLI or ISE but through the Hypervisor itself.
Anyway ... whatever the case with that ... I'm going to see if I can get this Nano to play nicely with my Hypervisor. I guess they didn't give you a port (and services) list. No worries ... I just need my :hammer: and :wrench: to get it working!

"I'm going to see if I can get this Nano to play nicely with my Hypervisor".
@GuardRex Could you elaborate on how you are blocked?

I was blocked too. Follow the 3 minute video https://channel9.msdn.com/Series/Nano-Server-Team/Demo-Building-a-Nano-Server-VHDX-for-a-Hyper-V-VM to get it working. The nano team claims the instructions at https://technet.microsoft.com/en-us/library/mt126167.aspx also work.

@shirhatti @Rick-Anderson Apparently, the Guest Drivers package doesn't open all of the ports if DISM is used on a running Azure Nano. I had to manually open ...
Virtual Machine Monitoring (RPC)
Virtual Machine Monitoring (NB-Session-In)
Virtual Machine Monitoring (Echo Request - ICMPv6-In)
Virtual Machine Monitoring (Echo Request - ICMPv4-In)
Virtual Machine Monitoring (DCOM-In)

@Rick-Anderson The VHD(X) built image(s) seem to get the right ports open by default. The Azure image-generated Nano seems not to have them open. I even tried adding the SCVMM package to see if that had a good effect, but it didn't.
... and the exception Hypervisor is giving me is ...

Hyper-V encountered an error trying to access an object on computer 'NANODEMO' because the object was not found. The object might have been deleted, or you might not have permission to perform the task. Verify that the Virtual Machine Management service on the computer is running. If the service is running, try to perform the task again by using Run as Administrator.

That's as far as I got last night, and I just sat down to pick up from there, so I haven't done any additional troubleshooting yet on that one ... but at least the Hypervisor sees and is trying talk to the Nano now.

@shirhatti I guess I should also note that I'm not doing this an officially "MS supported" way, thus what I'm doing now can't go into the doc. Azure VM's don't support Hypervisor. You can't use Roles & Features to install the tools. You have to manually do it in PS. That's why this is more a "for Luke's fun and use" than a "very helpful for the doc" kind of thing.

@shirhatti I found the complete list of ports for Server Manager AFAIK ...

Remote Service Management (NP-In)
Remote Service Management (RPC)
Remote Service Management (RPC-EPMAP)
Remote Event Log Management (NP-In)
Remote Event Log Management (RPC)
Remote Event Log Management (RPC-EPMAP)
Windows Firewall Remote Management (RPC)
Windows Firewall Remote Management (RPC-EPMAP)

... still getting ...

Error nanodemo : BPA result refresh failed with the following error: The system cannot find the file specified. 	1/6/2016 1:47:15 AM
Error nanodemo : Performance counter refresh failed with the following error: Class not registered 	1/6/2016 1:47:15 AM

I guess those are totally expected right now ... biz practices analyzer and perf counters ... under development I guess.
I'm switching over to look at the Hypervisor bits now (got slowed down by some company ops stuff this afternoon). Thus far, I see some Events that look related to this Hyper-V problem:

The VM and host networking components failed to negotiate protocol version '6.0'
The VM and host networking components failed to negotiate protocol version '6.1'

@Rick-Anderson @shirhatti I'll pick back up with my unsupported-by-MS :smile: Hypervisor scenario later. Bigger :fish: to fry now ...
Big doc update tomorrow. I'll ping here and by E-mail when the updated doc is ready sometime Wednesday afternoon.

@Rick-Anderson @shirhatti The doc has been (mostly) updated with a major emphasis on security. Instead of having folks go direct to the Nano Server via 5985/5986 over the open Internet :skull:, the doc has been revised for the following ...

Added sections and notes dealing with security issues with links to starting points for learning more.
Create a 2012R2 Management VM (so there are a pair of VM's now).
Have the management VM and Nano on a VNET.
Create Network Security Groups for the VM's.

2012R2: Opens only RDP (3389).
Nano: Opens only HTTP (80) & HTTPS (443) to the Internet. Opens 5985 on the VNET.

Eliminated the PSEdit example, as 2012R2 comes out-of-the-box with PS 3.
Utilized a set of prelim steps in Azure to establish the Resource Group, Storage Account (and File Share container), the Virtual Network, and the Network Security Groups for use later in the tutorial when deploying the Azure VM's.
Have the folks also create and use static IP addresses and set a DNS address label on the Nano for easy reference.

This 2nd draft is almost done. I still need to carry the DNS address label down through the Troubleshooting section (Server Manager and remote Event Log). I should have that done by Friday noon. You can probably look at this now ... you'll just see IP addresses in the Troubleshooting section that will become "nanodemo1234" by Friday.
As I was making these changes today, it didn't escape my attention that this does make the tutorial substantially heavier. However, may I remind you of the ... let's call it ... The @blowdart Gambit! That's a sacrifice of time and energy initially for a great advantage in security later. I prefer this approach and cross-introduction to a few good security concepts while demonstrating the amazing Nano Server. "Small in size. Big in :heart:!" However, your call, of course.
http://rexpublic.blob.core.windows.net/public/ASPNET5-on-Nano-Server.zip

Wait, are you calling me heavy?
More seriously I'm pretty sure everyone knows I would like everything secure by default, even if it brings more work. Otherwise folks will take the easy route, if you present it to them, regardless of security implications/

@blowdart :+1: ... and the easy route would be to make the tutorial simple and fast but almost encourage folks to leave 5986 open publicly (and other ports for Server Manager and remote Event Viewer ... Event Viewer being virtually required for .NET work), when that's vastly less secure and a violation of best management practices.
There's no easy way around it. If we want management points locked down, then we have to take the time in the tutorial to show it. If they go off-road and set the Nano up for direct access, or even worse use it publicly over 5985, that's on them. I even wanted to use a non-standard port for RDP, except that NSG doesn't do NAT, and we don't want them changing the port in the registry ... had to draw the line somewhere. The tutorial obviously can't be a book.
I'm looking forward to feedback. I'll get rid of the Nano IP address in Troubleshooting (in favor of the Nano's DNS address label) within 48 hours and ping back when that's done.

I appreciate the work going into this.
Honestly.

@GuardRex shouldn't
New-NetFirewallRule -Name "DCOM-In_135" -DisplayName "DCOM Service Control Manager (DCOM-In)" -Protocol tcp -LocalPort 135 -Action Allow -Enabled True

As its an existing rule, be something more like
Set-NetFirewallRule -DisplayName "Windows Remote Management (HTTP-In)" -Enabled True

Not sure what the right rules are but you can get them with
> Get-NetFirewallRule

@benaadams In the latest draft, I only have Net-NewFilewallRule for opening port 8000. There is a handy one for the existing rules ...
Enable-NetFirewallRule -DisplayName "<DISPLAY_NAME>"

... which I found works better than using the -Name property on that command (for whatever reason).
If you haven't seen the latest draft, I'm still working on grammar and phrasing, but you can get it at http://rexpublic.blob.core.windows.net/public/ASPNET5-on-Nano-Server.zip

Also you can use the PS command
> Restart-Computer

rather than doing it in the portal

@benaadams thx ... change made.
@Rick-Anderson @danroth27
2nd Draft Final is done (when you open, confirm by looking at the title, which states "2nd Draft Final"). I'm going to run through it all again start-to-finish from scratch on Saturday morning. I'll ping here if anything goes sideways.
http://rexpublic.blob.core.windows.net/public/ASPNET5-on-Nano-Server.zip
Let me know as soon as you can if you want to take this approach or go back to the low-security-one-VM-5986-direct (with new Nano cert) version. I'd like to start refactoring into RST asap, but I'll wait until you make a final call on the approach before I do the deed. :wrench:

1/8 10:13pm PST ... Minor update regarding use of cmdkey to persist the credentials for the network file share across reboots.

1/9 11:06am PST ... Minor update: Removed static public IP address assignment step for the Nano as unnecessary. Added a note as to why the management VM is getting a static public IP address [it's merely a convenience for using Remote Desktop Connection (RDP) across deallocating/re-allocating the management VM].

For the 3rd draft, I want to change the name of the storage account to "storage1234," because I think naming the storage (currently "nanodemo1234") the same as the Nano ("nanodemo1234") but also using that storage for the management VM ("vmmdemo1234") is a bit confusing. I also think that using the word "nano" in the shared services isn't good.
How about a scheme like this:

Resource
Name

Nano Server VM
nano1234

Management Server VM
vmm1234

Resource Group
group1234

Storage Account
storage1234

Virtual Network
vnet1234

@Rick-Anderson I've updated the text and images to conform to the naming in my last comment. It's definitely an improvement.
http://rexpublic.blob.core.windows.net/public/ASPNET5-on-Nano-Server.zip
Unless you have any objection, I'm going to move it to RST now.

@Rick-Anderson It's going well with RST. I'll have it done today. I have one more minor update to apply: For convenience, I used the same username for the management VM as for the Nano. Since the tutorial takes its time on security issues, I want the management VM on username "adminuser" and the Nano on username "nanouser."
Note that I'm pulling the direct link to the Word version out this issue. There have been a number of minor updates going into RST that make the Word version obsolete (especially those horribly long section and sub-section titles that I had).

@Rick-Anderson @danroth27
any chance to update the nano tp5 tutorial to RTM?
are the workarounds still necessary?

@rs38 Yup, we will be doing a general content refresh once we have completed the migration to docfx. It would be great if you could file a new issue specifically for updating the Nano article.


DOC: Add show-inheritance

Codecov Report

Merging #326 (3fc392c) into main (0207be5) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #326   +/-   ##
=======================================
  Coverage   99.87%   99.87%
=======================================
  Files          46       46
  Lines        1582     1582
=======================================
  Hits         1580     1580
  Misses          2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0207be5...3fc392c. Read the comment docs.


Enable clicking on the dots at the bottom of the slide

I your options object specify paginationClickable: true.
Check out the section on pagination in the docs, here.

Thanks. Could this be enabled in the online demo? (I should've been more specific in the issue title)

@dandv it will not be enabled in online demo. It is well described in documentation and presented in a lot of demos, for example in this https://github.com/nolimits4web/Swiper/blob/master/demos/02-responsive.html

Very well. I do hope you realize that less sophisticated users who evaluate slider libraries, might be turned off by the apparent bug of the dots not doing anything when clicked, and leave for a competing plugin.


Test credentials

Useful for adding end-to-end tests, e.g. that both client.get('statuses/show/964185606803853314') and client.get('statuses/lookup', {   id: '964185606803853314,965299128162029568',   map: true, }) return the same tweet. (These are tests I had run locally before my #10 PR).

:wave: hey this is an awesome idea! @dandv
Do you have any idea how to set this up without having a nefarious individual steal the vars?

Created test credentials and added them to travis. Great idea, thanks!


Update interop test service url to grpc-test.sandbox.googleapis.com.

FYI to @jtattermusch, @ctiller, @nicolasnoble

Can one of the admins verify this patch?

Can one of the admins verify this patch?

LGTM

I will wait a bit for the test before merging.

The test won't work until the update on GFE configs go live. I'll try locally when it does and post the result here. For now, please check if I miss anything.

I think all the occurrences I can find are fixed.

Please be careful with running interop tests suite on this pull request. As the  grpc-test.sandbox.googleapis.com service is not working yet, the test run can take extremely long time because of retrying all the cloud to prod tests several time (and sometimes timing out). Such long test run can prevent interop tests runs for other pull requests from being processed.

I am planning to merge this now. Any objections?

Just verified that grpc-test.sandbox.googleapis.com works.


Improving street resurfacing data calculation

The current street resurfacing scripts find the last time a street segment (section of street between intersections) was resurfaced. However, sometimes only a portion of a street segment is resurfaced at a time, which means we might be missing old sections of street because they're part of the same segment as a recently resurfaced section of street.
Changes to make:

Write a function that takes a street address, interpolates the distance along a street segment, and splits that street segment into two street segments.
Update street resurfacing script so, for a given resurfacing row, the street segments that the resurfacing starts or ends in are split using the function from step 1 (segments contained in the middle of the resurfacing can be kept whole).

I can work on this


[Bug Report]: Enchantments not working

Spark link
No response
Server Version
1.20.2
Server Platform
Paper
Describe the bug
AcidRain, Leadership, Intimidate, Blizzard, Radiant, Self Destruct and SandStorm don't seem to work at all.
Steps to reproduce
Apply the enchants on armor and try them out, they don't work.
Additional context
No response
Agreements

[X] I have searched for and ensured there isn't already an open issue regarding this.
[ ] I ticked all the boxes without actually reading them
[X] My version of Minecraft is supported by CrazyCrew.

Problem solved, Thread Closed.


Fix bundling and optimization

Setup webpack or gulp to export bundle with all dependencies.
This has to be done to enable bootstrap server-side rendering.

Help reference

Fixd in f9f3b28b0529e05693a3163a17a544c78484abc0.


Suggest some new features

Would love to see your imagination so just lmk if you have any new ideas to add in the game !

Add some new shapes, arenas and a special ability for each shape to spice it up a bit(Like a shape's special ability can be shield and another's can be thrust)but just for a short period.

@MahirMahdi sounds great, let's take one at a time, what would you like to see first ? and would really appreciate if you can create an issue for it !

Yes sure. Let's take one at a time then. 😀
Let me create an issue for the feature that I want to see first, then we can discuss about the other features after working on it.

I just played the game by myself with both hands and found it quite fun. However, I guess if I played it with another person, it might be endless.
I think adding features that could easily end the game, such as a jump feature, would be beneficial. For example, when the opponent is pushing you to the edge, you could jump to make them fall off the edge.

@iburn78 so by that you mean making it online multiplayer right ? actually we can do that, few weeks ago we built https://sharktanks.tech/ which is an online multiplayer game so that could be a great idea I believe.
jumping sounds good to me, would you like to create a new issue for it ?

@KlausMikhaelson Yes, eventually it could be an online multiplayer game. https://sharktanks.tech/ also looks fun. =)
Let me create a new issue for adding a jump feature in the game.

Hi, I have just played the game and it is great.
One feature that is coming to my mind is that there should be a timer so that one can know in how much time they lost the game.

After reviewing the "wrestle-with-shapes", here are some potential new features that could be added to enhance the functionality and user experience:

Shape recognition: Implementing shape recognition would be a useful addition, allowing users to identify the type of shape that they are working with. This could be particularly helpful when working with more complex shapes, where it might be difficult to distinguish between different types of shapes.
Interactive interface: Currently, the library is used via code. Adding an interactive interface, like a graphical user interface (GUI), would make it more accessible to users who are not comfortable writing code. This could also enable more complex operations to be performed, like grouping and ungrouping shapes, or dragging and dropping shapes.

Adding these features would enhance the functionality and versatility of the "wrestle-with-shapes", making it more useful for a wider range of applications.

You could add a Hit feature where the shape can dash into the other shape making the other shape fly away


⚠ Bofrost.nl has degraded performance

In 6a8cdd4, Bofrost.nl (https://www.bofrost.nl) experienced degraded performance:

HTTP code: 200
Response time: 931 ms

Resolved: Bofrost.nl performance has improved in 4aaac27.


Login always prompts for user interaction in this version of angular wrapper

Hi,
We recently update our auth to use @azure/msal-angular from adal-angular but we have an issue to bypass login prompt and auto login which used to happen in adal.
We did some research and we think msal-angular uses an older version of msal-js library.
Is there a plan or Can we update the msal version to newer 0.2.4 version? https://github.com/benbaran/msal-angular/blob/master/package.json#L25.
https://github.com/AzureAD/microsoft-authentication-library-for-js/releases , this release has the fix for the issue I guess.
Thanks

Hi @pnaika  since this package is not maintained anymore, and @azure/msal-angular does not yet fit all my requirements I created my own package using<EMAIL_ADDRESS>and which I will maintain until @azure/msal is final and mature. You could try it: npm i angular-msal: https://github.com/Marcelh1983/angular-msal


Getting build error with Xamarin.Build.Download 0.10.0

I am migrating to AndroidX and downloaded the latest version 0.10.0 but in android i am not able to build the project .
Its giving me this error . Error MSB4064: The "AndroidFixManifests" parameter is not supported by the "XamarinDownloadArchives" task. Verify the parameter exists on the task, and it is a settable public instance property.
Error MSB4063: The "XamarinDownloadArchives" task could not be initialized with its input parameters.
Please let me know if anybody is facing this issue . If there is any solution i am thinking its an issue in latest version.

I'm having the same issue with Xamarin.Build.Download 0.10.0
Severity	Code	Description	Project	File	Line	Suppression State
Error		The "AndroidFixManifests" parameter is not supported by the "XamarinDownloadArchives" task. Verify the parameter exists on the task, and it is a settable public instance property.

Having the same issue

One developers in our team has this issue as well.
The difference is that he is the only developer on this project who is on Windows, the rest is on macOS.
None of the developers on mac have this issue.
Any clues?

Anyone solve this? im getting it as well...

Me too, and it seams to come out of nowhere.

Probably caused by this https://github.com/xamarin/XamarinComponents/pull/1368
Because here the AndroidFixManifests was removed.
Workaround: referencing a previous Version:
Adding following to the Android Project solves this (uses Xamarin.Build.Download Version that has still this Attribute)

0.11.1

I get this error while building the iOS project. Very strange.

I get this error while building the iOS project. Very strange.

Same here, only started recently.
This only occurs when I build with macOS, on Windows everything builds fine. I've just disabled the Android build for Release builds of my Xamarin app for iOS in order to be able to publish iOS builds.

I get this error while building the iOS project. Very strange.

Me too. Got the error building for iOS.

It is very annoying. Switching between Debug|Phone and Debug|Simulator sometimes fixes it. Or closing re-opening VS. But suddenly this error is back again. I think it started when I installed the pre-release for 17.4 (which I now delete) and am using stable version 17.4.

I had this error due to Xamarin.Build.Download missing from NuGet.

Fixed by opening Tools->Nuget Package Manager->Manage  NuGet Packages for Solution...
On the Browse tab, search for Xamarin.Build.Download
Ensure it is installed for all of the projects and that they are the same version (0.11.4)
Then Clean Solution followed by Rebuild Solution.

Thank you @johnrah, you're a legend! Installing the Xamarin.Build.Download package fixed it for me.

Just encountered this issue when upgrading my .Net Maui Android app to .Net 7.0. @johnrah suggestion fixed the problem.

@johnrah suggestion fixed the problem. Thanks a million!

This solution work for Maui net6.0 project. Thanks!

Thanks @johnrah

Hey thanks @johnrah, I was getting this out of the blue on a Maui net6.0 project.  It was strange, as I was trying to build for iOS at the time.

nice one. Worked for me too, this nuget package.

Thanks @johnrah that helped me

I had this error due to Xamarin.Build.Download missing from NuGet.

Fixed by opening Tools->Nuget Package Manager->Manage  NuGet Packages for Solution...
On the Browse tab, search for Xamarin.Build.Download
Ensure it is installed for all of the projects and that they are the same version (0.11.4)
Then Clean Solution followed by Rebuild Solution.

Thanks, it works now.

@johnrah what is Xamarin.Build.Download and why can I build my .NET MAUI app once and it fails 2mins later - without any code change? Just curious...

@johnrah what is Xamarin.Build.Download and why can I build my .NET MAUI app once and it fails 2mins later - without any code change? Just curious...

Xamarin.Build.Download provides support for downloading and unpacking archives when building MSBuild projects
https://www.nuget.org/packages/Xamarin.Build.Download/
As for the reliability of .NET MAUI, it is very fickle. It's getting better with every update, though.

OMG this saved my day big time. Thank you!!

Thanks This saved may day, today not the last two ones. solving some cryptical build and deployment (IOS) errors ends up with a new one and so on. Spend the last two days without writing one line of code. Its frustrating. I was a great fan of xamarin forms .  But now as a SW Architect I cannot suggest using Maui for real live projects.
@johnrah dotnet Maui is still very fickle.

Thanks @johnrah! This stopped the sporadic build issues

I have move on to Flutter from last 1 year.
Didn't got any issues around packages, builds. very very fast for development.

I am still having this issue and like others said strangely only happen to iOS builds (Windows, Android and Mac Catalyst build without any issue). Adding the package even only for iOS (with Condition attribute) fixes it.
This happens when building on a Windows with or without Pair to Mac.

I don't know when or what happened but after a recent update, Xamarin.Build.Download is probably added back to the default iOS build. I now receive a duplicate package error until I remove it and I can build iOS apps normally without that package now.


Install Problem on OsX Monterey 12.6

brew install ddelange/brewformulae/yt
==> Tapping ddelange/brewformulae
Cloning into '/usr/local/Homebrew/Library/Taps/ddelange/homebrew-brewformulae'...
remote: Enumerating objects: 143, done.
remote: Counting objects: 100% (95/95), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 143 (delta 21), reused 84 (delta 13), pack-reused 48
Receiving objects: 100% (143/143), 31.70 KiB | 5.28 MiB/s, done.
Resolving deltas: 100% (29/29), done.
Error: Invalid formula: /usr/local/Homebrew/Library/Taps/ddelange/homebrew-brewformulae/Formula/spek.rb
spek: undefined method `cask' for Formulary::FormulaNamespace3bbcb78e93dd3039825a3b792493d41b:Module
Did you mean?  case
Error: Cannot tap ddelange/brewformulae: invalid syntax in tap!

ref https://github.com/ddelange/homebrew-brewformulae/commit/cfb1d0a4d2a2f1df284346b5df3843779ba9bf24
can you try now?

hi @sbonaime 👋
did the installation succeed?

please comment here or open another issue if you still experience problems!

Well... I upgrade my system to Ventura and I need to re-install Xcode but I am running out of space ! So I don't know if the fix is working...
Thanks


Add a coflatMap method to traversable types

Useful for user defined X[A] -> X[B] operators.

Merged


Netapp storage failover

Type, Provider, Unit tests, Acceptance tests written for netapp storage failover command. Kindly review.

Hello Hunner,
I have modified code as per the situation you mentioned.
Can you please see it once?


Seeing slab_reassign_evictions_nomem when items move to SSD

We are actively using ExtStore feature and we see evictions (slab_reassign_evictions_nomem) during long running data ingestion jobs.
In many of our deployments we hit the limit of available RAM. These are big VMs with 112G of memory and 800G of SSD. Our observation is that during long running ingestion jobs items get shuffled causing some items in RAM to be moved to SSD. We have enough free space on SSD and we expect no evictions - items to be simply moved from RAM onto SSD. However, at some point we start seeing evictions (coinciding with slab_reassign_evictions_nomem metric). With slab_automove_freeratio defaulted to 1%, the rate of evictions becomes very high (thousands per sec). When we set slab_automove_freeratio to 20%-25%, evictions get postponed and happen occasionally with a considerably lower rate (tens per sec).
It might be a speculation at this point, but we think what happens is that when an item needs to be offloaded onto SSD, its metadata gets moved from a higher to a lower slab. When the lower slab does not have space to grow, it simply evicts one of the other LRU keys (apparently, another metadata for another SSD-offloaded item).
No matter how high we set slab_automove_freeratio (up to 40%), there is still a point when evictions start happening and our customers complain.
Any thoughts/suggestions on how to eliminate or mitigate these evictions would be greatly appreciated.

Have you tried manually setting ext_item_age to some low level? That can greatly increase the aggressiveness of the flushing and help avoid slab page move scenarios that can lead to _nomem evictions.
I've had this thought for a while though, see here: https://github.com/memcached/memcached/issues/541#issuecomment-619457044
It's a few significant steps but entirely removes _nomem as a possibility and would simplify a few related systems.
If your co's up for a work contract I can see about prioritizing that change higher or doing more detailed analysis on what's going wrong.

Thanks for the ext_item_age hint, @dormando! Will try it. I believe we should be able to set it without restart required. #541 looks promising but involved. I doubt about the contract, but will ask.

@dormando, setting ext_item_age does help with the heavy-ssd-writes case, thanks again for the hint.

Great to hear! I agree it does feel lame. ext_item_age was the original flush method while I was developing extstore, and I only left it in for compatibility; but then other folks ended up using it. I would be curious to know if you can actually measure a difference in read performance, and how significant that is if so.
Also for what it's worth, if you reduce the size of the cache daemon a bit (or if it hasn't allocated all of memory) then the OS gets more filesystem buffer space, which is used by extstore presently. For items recently flushed to SSD you'd still get them pulled back without hitting disk, just a slightly slower path internally to memcached.
Another thing is some folks will set ext_item_age while doing a bulk/fast load, then disable it once the bulk load is done. Since bulk loads have no information on the value of what should actually be in memory, letting that resolve once the bulk load is done can help (via tuning the recache-rate if necessary).

Can't give you much more trusted data at this point. We are a kind of "service-provider" team within a very large org providing memcached deployments with mcrouter proxies between the clients and the shards. We have limited visibility on customer flows.
Yesterday we set the age to 15 sec on one of the clusters and the RAM utilization went down to almost 0. There was a performance test and it looks like it held pretty well. The average latency on Mcrouter side increased twice but still held at half-msec on small payloads. And the max was still good enough. This includes a network hop from Mcrouter to memcached. We run on VMs and the cloud provider is capping the disk IO, though, the latency.

Understood. Also keep in mind the slab rebalancer algorithm can be run externally to the daemon (see scripts/memcached-automove*). You might be able to customize something that works better, possibly even by automatically adjusting ext_item_age when the global page pool gets too low (on top of the existing algo). There are some options that shouldn't be too painful while you wait.
RE: Proxy. We're still a ways off from being production ready, but I'm just about at the point of designing the route API. If you folks are interested in helping, taking a look would be much appreciated. There's a long list of further work but not everyone will need everything to go to production (or just to test), so any hints as to what's most important for you will help a lot.
Once I get through a pass on the API and more confidence in stability I can start documenting / blogging about it, which should make it easier to get into. You can get pretty far starting from the example configuration noted in the PR though.

https://github.com/memcached/memcached/issues/796 for proxy discussion.

https://github.com/memcached/memcached/pull/923 - this change might alleviate the need for ext_item_age = 0 in this use case, ahead of the planned larger rewrite that I've not had time for.


Python 2.7 Anaconda-VS Code. Not able to download tweets in csv

Hi,
I am tryin to use Exporter.py code in Anaconda - VS code with python version 2.7 . However, when I use
python Exporter.py
--username "barackobama" --maxtweets 1
the output_got.csv file is all empty.
Can anyone help me in what could have gone wrong.

Hi,
I am tryin to use Exporter.py code in Anaconda - VS code with python version 2.7 . However, when I use
python Exporter.py --username "barackobama" --maxtweets 1
the output_got.csv file is all empty.
Can anyone help me in what could have gone wrong.


Pod installed Alamofire 4.4, but class is not being autocomplete by Xcode 8.3

I have noticed this problem as closed before but I am getting the same problem.

P.S - I have imported Alamofire in the class mentioned in the screenshot
Here is my pod details
platform :ios, '10.0'

use_frameworks!
target 'PhotoTagger' do
     pod 'Alamofire'
end

Also the only thing being auto completed is AlamofireVersionNumber and i cannot access anything when I am trying to use the auto complete feature. Now if i write Alamofire by hand then I can access some methods, but then again something is going wrong and code is not compiling, which i think boils down to the issue that the class isnt being fetched by auto complete.
I have tried -
In Project settings > Build settings

Always search user path To YES
Header search path to only include "$(inherited)" as non-recursive

All other pods are working fine, except Alamofire.
Also after writing class name by hand or copy pasting, some of the methods that are being auto completed does not have the same signature of the actual method of the Alamofire class
Please please please !! help me sort this out, so that someone out there when facing this can help themselves too. There isnt any solution to this as of now.

import Alamofire

Sorry, we're not responsible for Xcode's autocomplete issues. In the future, you should open questions like this on Stack Overflow and tag alamofire.
Cheers. 🍻

From our Contribution Guidelines
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.

@jshier Thanks for stating the obvious.
But, this is a serious issue, related only to Alamofire, and I think this worth some investigation.
I'm having the exact same issue on multiple machines and it is happening ONYL to Alamofire. This might be an issue with the Podfile. rejecting a bug report because it seems like a support request is the easies way.
P.S.: High Sierra (10.13.4), Cocoapods (1.5.2 and 1.5.0), Xcode 9.3

@mohpor, there are thousands of developers currently using Alamofire with the latest releases exactly as you just posted (myself included) with no issues. The issue IS NOT Alamofire or the podspec, but something about your machine set up. Our team does not have the bandwidth to troubleshoot Xcode and CocoaPods instability issues for every developer that runs into this type of problem. Please understand that. As always, the following tend to be helpful:

Relaunching Xcode
Deleting DerivedData and the Xcode cache
Relaunching Xcode again
Rebooting your machine

Then if things are still not working, then you can try resetting up the project. Possibly reinstalling Xcode. Sometimes people have to re-image their machine. Just depends on how screwed up the system is. Sadly, this is just the day-to-day state of affairs of developing with Xcode.
Best of luck. 🍻


Removing trailing comma if applicable

Fixes https://github.com/uber/piranha/issues/23
Context: Incase there is a trailing comma in the ConditionElementListSyntax, the PR aims to re-write the syntax node by removing the trailing comma if applicable. Suitable test-cases are added to cover the rewriting part.

Makes sense, added test-cases asserting multiple condition behaviour.


keedump: init at 0.1.0

Description of changes
KeePass master password dumper
https://github.com/ynuwenhof/keedump
Related to https://github.com/NixOS/nixpkgs/issues/81418

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 268586 run on x86_64-linux 1

1 package built:

keedump


add aniketartani

Fix #2 Please accept my PR @iamdevvalecha

@iamdevvalecha please merge it

Thank you for contributing! This PR is successfully part of Hacktoberfest! 🎊


Clarify provided geo-coordinates are AWS-only, make example match them

We should follow-up with coordinates for the GCE and Azure regions, but
these are not correct for anything other than AWS - for example, AWS's
us-east-1 is in Northern Virginia, while GCE's us-east1 is in South Carolina.

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/eceb6b616752c7984c37253540941dbce73347e1/

Rebased and fixed merge conflict.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/a671b54df5674e137ca68c142a0be8d521502bf9/

@a-robinson, did you see @Amruta-Ranade's comment? Seems we need to resolve that before moving forward here.

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/da831da1d229fbfb0aed1884a6e3b8868bf7750a/

To support my request that we add GCE coordinates, we got this docs feedback submission from Tierion just last last night:
Submitted April 5, 2018 6:45 PM (0 days ago)
Improvements
Something's broken (links, etc.)
What's broken?
The text has a link for "the latitudes and longitudes of common AWS/Azure/GC regions, see Locations Coordinates for Reference."
But the table at the bottom of the page has only locations for AWS. Values for Google Cloud would be helpful.
Email
<EMAIL_ADDRESS>cc @Amruta-Ranade

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/6f9f38faf620b8e1c798932401fa7615a902415f/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/7d35194f9327adf1f5a483ec983aeacca3a0ec83/


Contribution showing use in a digital clock.

Using 7-segment displays, here's digital clock:
https://editor.p5js.org/Khaley4/full/By3E58MsX

Accepted suggested change.  Do I need to do anything else?
...Ken
On Fri, Oct 19, 2018 at 10:22 AM Nils Weber<EMAIL_ADDRESS>wrote:

@nilsweb requested changes on this pull request.
In _CodingChallenges/117-seven-segment.md
https://github.com/CodingTrain/website/pull/783#discussion_r226685660:

@@ -75,7 +75,12 @@ contributions:
author:
name: "The Tasteful Toastie"
url: "https://github.com/TheTastefulToastie"

url: "http://jsfiddle.net/TheTastefulToastie/qm1089v6"

url: "http://jsfiddle.net/TheTastefulToastie/qm1089v6"

There's some trailing whitespace here.
⬇ Suggested change

url: "http://jsfiddle.net/TheTastefulToastie/qm1089v6"

url: "http://jsfiddle.net/TheTastefulToastie/qm1089v6"

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CodingTrain/website/pull/783#pullrequestreview-166573985,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYnPLwCxkUlfl2YnT2coxJERSqTgQc6pks5ume5QgaJpZM4XwNiZ
.

Thanks! 🎉


Exceptions?

Exceptions?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0cb433c5-8e9c-389c-6705-cbb7db7ee7fb
Version Independent ID: 086349c2-0a79-4ebf-0069-105fe9720739
Content: StorageItemAccessList.GetFileAsync Method (Windows.Storage.AccessCache) - UWP app developer
Content Source: https://cpubwin.visualstudio.com/DefaultCollection/windows-uwp/_git/winrt-api-build?path=%2fwinrt-api-build%2fwindows.storage.accesscache.storageitemaccesslist.yml&version=GBlive&_a=contents
Product: windows
Git Login: wdg-dev-content
Microsoft Alias: wdg-dev-content

Thanks for taking the time to let us know about this and we sincerely apologize for the delayed response. Due to the sheer volume of feedback in our repo, we are currently prioritizing new Issues. If you believe your feedback is still actionable, let us know either directly in this thread or by filing a new Issue and we’ll work to incorporate the suggestions into the documentation. Though the Issue will be marked as Closed, I will monitor any replies that come afterwards so feel free to treat this like an open Issue. Many thanks!


expando fields not added to symbol exports

TypeScript Version:  3.4.0, 3.5.0, master (7dc1f40dc15132ba87a70a3bec6d63317cb5b91e)

Search Terms:
expando, iife, umd, exports
Code
Given the following code, similar to an UMD file:
(function(){
    var A = (function () {
        function A() {}
        return A;
    }());
    A.expando = true;
}());

Expected behavior:
The ts.Symbol of the outer A should have expando in it.
Actual behavior:
The ts.Symbol of the outer A does not have expando in it. When the declaration of A is at the top-level, without the "UMD wrapper", it works properly:
var A = (function () {
    function A() {}
    return A;
}());
A.expando = true;

I have researched what is preventing the expando static prop to be added to exports, and it is due to when a ts.Symbol is considered for expando properties, here:
https://github.com/microsoft/TypeScript/blob/3d2af9ff332fca6c5db2390be0b1f08bba8402a1/src/compiler/binder.ts#L2649-L2678
Specifically, the symbol's flag of A inside the UMD wrapper are not sufficient to take the early-return in the first statement, whereas with A at the top-level it has been assigned appropriate flags.
Because the appropriate flags are not present, the code structure is analyzed to determine if the symbol should be classified as expando. Specifically, the relevant code is in getExpandoInitializer:
https://github.com/microsoft/TypeScript/blob/3d2af9ff332fca6c5db2390be0b1f08bba8402a1/src/compiler/utilities.ts#L1873-L1896
From this function, it becomes clear why the symbol fails to be recognized as expando symbol: the used IIFE syntax deviates from the syntax that is accounted for in getExpandoInitializer. When changing the sample into the following, it does indeed work as expected:
(function(){
    var A = (function () {
        function A() {}
        return A;
    })(); // <-- The difference is here
    A.expando = true;
}());

Unfortunately however, downleveled ES5 code does use the syntax that is not accounted for.
Playground Link:
Can't be replicated in the playground, but copying the code into https://ts-ast-viewer.com, setting the script kind to JS and inspecting the Symbol of the outer A declaration shows that its exports are empty. The alternative IIFE syntax does have the proper exports.
Related Issues:
n/a
Background info
This is an issue for Angular's Compatibility Compiler, which uses the TypeScript compiler to parse JavaScript bundles in various formats and uses the symbol information to reason about the code. PR https://github.com/angular/angular/pull/30795 now contains a hack to patch TS's getExpandoInitializer, which does indeed resolve the issue.

I tried to create a testcase tests/cases/conformance/salsa/typeFromPropertyAssignment38.ts to replicate the issue, however the following didn't work out:
// @noEmit: true
// @allowJs: true
// @checkJs: true
// @noImplicitAny: true
// @Filename: a.js
(function(){
    var A = (function () {
        function A() {}
        return A;
    }());
    A.expando = true;
    exports.A = A;
}());

// @Filename b.js
var a = require('./a');
a.A.expando;

It has the exact same golden files compared to the alternative IIFE syntax.

@sandersn should we take a PR to drill through IIFEs?


Longer days mod/cheat?

Wondering how possible it is to make mods/cheats for the day duration? I think default is way too quick. I've had a look through the dump files to see if it was moddable but no luck.

I don't know sorry, did you searched on gamebanana ?

I wrote an IPSwitch patch to do this on 1.1.0. I haven’t encountered any negative side-effects, but I have no idea what I’m doing, so use at your own risk, no warranty, etc.
Place in timeofdaymod/exefs/1.1.0.pchtxt:
@nsobid-D5AD6AC71EF53E3E52417C1B81DBC9B4142AA3B3

@flag print_values
@flag offset_shift 0x100

@enabled
00af4b64 3cc46314
023e5c54 390000d0
023e5c58 205f4cbd
023e5c5c a6000014
023e5ef4 0819201e
023e5ef8 0129211e
023e5efc 56170014
023ebc54 79a10090
023ebc58 c4239c17
023ebc5c 00000041
@stop

The last byte is a little-endian IEEE754 floating point number that respresents a multiplier for how much slower the in-game clock progresses. The above version (0x00000041) is equal to 8.0, so each day lasts 8× longer, or 3h 12m total (96 minute days, 96 minute nights).
Other possible values (I haven’t test any of these):

0x0000003e 8× faster (3 minute day/night cycle)
0x0000803e 4× faster (6 minute day/night cycle)
0x0000003f 2× faster (12 minute day/night cycle)
0x0000c03f 1.5× slower (36 minute day/night cycle)
0x00000040 2× slower (48 minute day/night cycle)
0x00002040 2.5× slower (60 minute day/night cycle)
0x00004040 3× slower (72 minute day/night cycle)
0x00008040 4× slower (96 minute day/night cycle)
0x0000a040 5× slower (2 hour day/night cycle)
0x0000c040 6× slower (2h 24m day/night cycle)
0x0000e040 7× slower (2h 48m minute day/night cycle)
0x00000041 8× slower (3h 12m minute day/night cycle)
0x00001041 9× slower (3h 36m minute day/night cycle)
0x00002041 10× slower (4 hour day/night cycle)
0x0000f041 30× slower (half “realtime”, 12 hour day/night cycle)
0x00007042 60× slower (“realtime”, 24 hour day/night cycle)

I make no copyright claim whatsoever over any of this. Do whatever you want, repost to gamebanana, make it better, etc:
This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.

In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.

IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <https://unlicense.org>

Have been testing it for a bit, no issues so far! Thanks so much dude, and for the guide. I'll upload it to Gamebanana and credit you if that's cool?

Glad it’s working for you! I’ll try to update it for 1.1.2 soon, got a lot on my plate at the moment but hopefully sometime in the next week?

I'll upload it to Gamebanana and credit you if that's cool?

Cool with me! There’s an open request here, feel free to claim the bounty or whatever, I don’t have a gamebanana account.

I wrote an IPSwitch patch to do this on 1.1.0. I haven’t encountered any negative side-effects, but I have no idea what I’m doing, so use at your own risk, no warranty, etc.
Place in timeofdaymod/exefs/1.1.0.pchtxt:
@nsobid-D5AD6AC71EF53E3E52417C1B81DBC9B4142AA3B3

@flag print_values
@flag offset_shift 0x100

@enabled
00af4b64 3cc46314
023e5c54 390000d0
023e5c58 205f4cbd
023e5c5c a6000014
023e5ef4 0819201e
023e5ef8 0129211e
023e5efc 56170014
023ebc54 79a10090
023ebc58 c4239c17
023ebc5c 00000041
@stop

The last byte is a little-endian IEEE754 floating point number that represents a multiplier for how much slower the in-game clock progresses. The above version (0x00000041) is equal to 8.0, so each day lasts 8× longer, or 3h 12m total (96 minute days, 96 minute nights).
Other possible values (I haven’t test any of these):

0x0000003e 8× faster (3 minute day/night cycle)
0x0000803e 4× faster (6 minute day/night cycle)
0x0000003f 2× faster (12 minute day/night cycle)
0x0000c03f 1.5× slower (36 minute day/night cycle)
0x00000040 2× slower (48 minute day/night cycle)
0x00002040 2.5× slower (60 minute day/night cycle)
0x00004040 3× slower (72 minute day/night cycle)
0x00008040 4× slower (96 minute day/night cycle)
0x0000a040 5× slower (2 hour day/night cycle)
0x0000c040 6× slower (2h 24m day/night cycle)
0x0000e040 7× slower (2h 48m minute day/night cycle)
0x00000041 8× slower (3h 12m minute day/night cycle)
0x00001041 9× slower (3h 36m minute day/night cycle)
0x00002041 10× slower (4 hour day/night cycle)
0x0000f041 30× slower (half “realtime”, 12 hour day/night cycle)
0x00007042 60× slower (“realtime”, 24 hour day/night cycle)

I make no copyright claim whatsoever over any of this. Do whatever you want, repost to gamebanana, make it better, etc. Any part of this that may be subject to copyright is hereby released into the public domain:
This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.

In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.

IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to <https://unlicense.org>

Awesome, we'll add it to the cheat section, except if you want to join the collaborators and do it yourself.

Some people on gamebanana were having issues with it. The 3h 12m version works for me on Yuzu and Ryujinx, game version 1.1.0. I don’t know if there’s something better than IPSwitch patches for this, if there is I can look into updating it.
Someone there also suggested it might not work with some FPS mods because they modify some timing code? I’ve only tested it with FPS++.

Yeah I've only used FPS++ too. I think it's a version thing, some people running 1.1.0 and 1.1.1 etc

Let me know when you get around to the 1.1.1 and 1.1.1 versions and I'll chuck them on GB. No rush man, cheers!

I added them, reopen the issue if you find the 1.0.0, 1.1.1 or 1.1.2 adresses.

Update for 1.1.2; I don’t have a copy of 1.1.1.
@nsobid-9A10ED9435C06733DA597D8094D9000AB5D3EE60

@flag print_values
@flag offset_shift 0x100

@enabled
00ae3150 99810014
00b037b4 590100f0
00b037b8 20df40bd
00b037bc 46110014
00b07cd4 0819201e
00b07cd8 0129211e
00b07cdc fe980014
00b2e0d4 196701b0
00b2e0d8 1fd4fe17
00b2e0dc 00000041
@stop

Possible time multipliers are the same as above; replace 00000041 with desired value.
Annotated version:
7100ae314c __ __ __ __     // Existing code of patched function
// Go to first code cave
7100ae3150 99 81 00 14     b          codeCaveA
    shimReturn:
7100ae3154 __ __ __ __     // Existing code of patched function

    codeCaveA:
// Our custom time multiplier is stored at 0x7100b2e0dc; overwrite x25 with the
// page address for our custom time multiplier
7100b037b4 59 01 00 f0     adrp       x25,0x7100b2e000
// Use page address at x25 to load the our custom time multiplier into s0
7100b037b8 20 df 40 bd     ldr        s0,[x25, #0xdc]
// Go to second code cave
7100b037bc 46 11 00 14     b          codeCaveB

    codeCaveB:
// Divide current time "slice" (s8) by time multiplier (s0)
7100b07cd4 08 19 20 1e     fdiv       s8,s8,s0
// Add time slice (s8) to current time of day (s1)
7100b07cd8 01 29 21 1e     fadd       s1,s8,s1
// Go to third code cave
7100b07cdc fe 98 00 14     b          codeCaveC

    codeCaveC:
// Restore original value of x25 so the patched function can use it
7100b2e0d4 19 67 01 b0     adrp       x25,0x710380f000
// Return to patched function
7100b2e0d8 1f d4 fe 17     b          shimReturn
// Custom time multiplier value stored here
7100b2e0dc 00 00 00 41

Added to last release.


Hardcoded /etc path

I am currently packaging sway for the NixOS distribution.
It manages configurations in a special way, preventing the user from writing in /etc in order to preserve a clean system.
In order to build sway on this distro, I had to patch CMakeLists to add a "DETCDIR" flag.
Therefore, a new /etc destination can be given at compile time (in my case, I give the one the distro wants in order to build sanely).
I would like to propose this change to be made upstream. Although it is quite rare to break the /etc convention, it is common for CMakeLists to expose such a parameter.

I would merge a PR that makes the path customizable.

Be right back then.


RDKB-51547 : [On rbusDirect] Implementation of RBUS Publish API to send Raw Data

Reason for change: Added new API for add and remove listener which will work based on the subscription ID. The existing API will remain to support the RDKC build. Also addressed the compile time issue faced for ccspcommonlibrary
Test Procedure: Build and verified
Risks: Low
Priority: P1
Signed-off-by: Gururaaja ESR<EMAIL_ADDRESS>

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 1 file pending identification.

Protex Server Path: /home/blackduck/github/rbus/187/rdkcentral/rbus

Commit: f7ab9cb4b26e44d004ee26764c71d50d7d73f212

Report detail: gist

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 1 file pending identification.

Protex Server Path: /home/blackduck/github/rbus/187/rdkcentral/rbus

Commit: a2f7b66aa49f3213c6a5afb5a973b1b97b0f23f9

Report detail: gist

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 1 file pending identification.

Protex Server Path: /home/blackduck/github/rbus/187/rdkcentral/rbus

Commit: 7351525dbf007f5919180196af25f5876c90fa88

Report detail: gist

WARNING: A Blackduck scan failure has been waived
A prior failure has been upvoted

Upvote reason: Not a problem

Commit: 7351525dbf007f5919180196af25f5876c90fa88


Alloy Smelter's Furnace mode doesn't work

Is there an existing issue for this?

[X] I did not find any existing issues.

Current Behavior
When in mixed/furnace mode, the Alloy Smelter won't start processing valid recipes. For example, it will not smelt Cobblestone to Stone.
Expected Behavior
Should process Furnace recipes
How to Test

Place an Alloy Smelter
Insert Capacitor
Provide Power
Insert Cobble
Nothing happens

Take the Cobble out, set to Furnace Only, Reinsert and it still doesn't go.
Local Environment
- **Operating System**: Windows 11
- **Java version**: 21.0.4, Eclipse Adoptium
- **Minecraft Version**: 1.21.1
- **Version**: 1.21.1+10
- **Other Mods**: Enigmatica 10 development branch

Any Additional Information?
No response

This is being tackled as we speak. #812 #820. Check #contributing on discord to track progress

Awesome thanks

Unable to reproduce this on 7.0.5-alpha, could you try again after updating the mod version?
Thanks!

Okay this works in a development environment but not when in a modpack like ATM10, something must be interfering with the mixin 🤔

Relentless discovered that this is caused by a KubeJS behaviour (see https://github.com/KubeJS-Mods/KubeJS/issues/822). We'll continue to work on a solution. Thanks!

This should now be resolved by commit 8a42d9dee9c770f7e24852838ae51a2df61474bb. It will be released shortly. Thanks!

It's better, but still a little funky :D
In Furnace Mode it will not smelt Cobble
In Furnace/Alloy Mode, it will smelt Cobble, but only if it has less than 3 input slots full.
In Alloy Mode it will smelt Cobble and only allow items in one slot.
https://streamable.com/f9lrk2

Yep that seems consistent with another issue I fixed earlier today - that'll be in tonights patch release. Thanks!


Loading mtl files for PartNet-Mobility dataset with Kuafu Renderer

Hi Fanbo @fbxiang,
I am interested in learning how to load the material files from partnet-mobility dataset and render them with the 3D model.
I looked up the kuafu_mat.py script, but it creates its own materials. What is the exact command to load .mtl file using the initialized renderer?

Also, I see that visual_body.type is not implemented for Kuafu Renderer.
Below is the output when I do this using Kuafu:
Traceback (most recent call last):   File "render_sapien_kuafu_cam_pos.py", line 194, in <module>     mesh = get_mesh_for_obj(obj)   File "../mesh_utils.py", line 84, in get_mesh_for_obj     print(visual_body.type) RuntimeError: getType is not implemented
FYI, the function is available for Vulkan renderer, and I was able to successfully render articulated 3D models.
How do I make use of this functionality with Kuafu renderer? @fbxiang
Thanks.

.mtl files are loaded in Kuafu automatically. You just need to load the .obj file.
Kuafu is also only designed to work as a renderer. Many convenient functions such as querying object properties (e.g. type) are not implemented as this point. For now, please use KuafuRenderer only for ray-traced rendering and use the default VulkanRenderer if you need more functionalities.

Thanks. The rendering quality from VulkanRenderer, in my case, are poor.
I am attaching a few images below, where you can see that the rendering of one partnet-mobility model is basically just color info.
Important image information such as edges, especially on the articulated parts, are not captured using Vulkan.
Is there a way to improve the rendering quality using Vulkan?

The default VulkanRenderer is based on rasterization, getting it to look nice is more like an art. Typically adding some lights and  an environment map help it to look better to humans (many algorithms may or may not care since it is not fully physical either way).
The KuafuRenderer (which is also implemented with Vulkan) is based on physically correct ray tracing, it is much slower but gives more photorealistic images with less effort.
You can debug your code with the default VulkanRenderer, and do a final rendering with KuafuRenderer.

Now sapien 2.2.0 is released and Kuafu is deprecated.
The original VulkanRenderer is now renamed  SapienRenderer and it supports all functionalities of Kuafu.
SapienRenderer's ray tracing is very slightly faster than Kuafu but a lot more stable.
You can now switch between rasterization and ray tracing without changing the renderer.