PyTorch

Speaking of Deep Learning's library Tensorflow is famous. Since it was released a year and a half ago, I got a name recognition for a moment.
I began looking for other libraries, thinking that it would be better to use a more flexible library than Keras. Then what is Torch's Python version out there!
DeepMind has moved from Torch to Tensorflow, but I anticipate that you are already using PyTorch. Facebook is also one of the representative companies using Torch, and I am involved in the development of PyTorch.

Machine learning result (mnist recognition)

Imaging mnist data

——mnist_data_image.py------
1 # -*- coding: utf-8 -*-
2
3 import sys
4 import numpy as np
5 np.random.seed(20160715)
6
7 from keras.datasets import mnist
8 from keras.utils import np_utils
9
10 (X_train, y_train), (X_test, y_test) = mnist.load_data()
11
12 for xs in X_train[1]:
13 for x in xs:
14 sys.stdout.write('%03d ' % x)
15 sys.stdout.write('\n')
16
17 print('first sample is %d' % y_train[0])
18
19 Y_train = np_utils.to_categorical(y_train, 10)
20
21 sys.stdout.write('[')
22 for y in Y_train[0]:
23 sys.stdout.write('%f ' % y)
24 sys.stdout.write(']\n')

PWA (progressive web application) supported on iOS

As I wrote in an article I posted in the past, PWA (Progressive Web Application) that I could only use in Chrome until now can be used in Safari.

The advantages of PWA are described below.

1. It is possible to provide user experience that is the same as or close to the application on mobile sites

2. Page loading is fast

3. Push notification is possible

4. By using the service worker mechanism, pages can be used even offline. FINANCIAL TIMES is an example.

OpenCV 3.1.0 install

$ sudo apt-get install build-essential cmake git
$ sudo apt-get install ffmpeg libopencv-dev libgtk-3-dev python-numpy python3-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff5-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev libtbb-dev qtbase5-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip

$ wget https://github.com/Itseez/opencv/archive/3.1.0.zip
$ cd opencv-3.1.0
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_V4L=ON -D WITH_QT=ON -D WITH_OPENGL=ON ..
$ make
$ sudo make install
$ sudo /bin/bash -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
$ sudo ldconfig

$ sudo pip3 install opencv-python

Machine learning self-made machine starting with low budget

◼️Hardware
GPU: 1050Ti ( CUDA core：678 , Memory : 4GB, Compute Capability : 6.1 )
PCI Express 3.0

◼️Software
(1) Ubuntu 16.04 install
(2) GPU Check
$ lspci | grep NVIDIA
(3) CUDA install
https://developer.nvidia.com/cuda-downloads
$ nvidia-smi
(4) cuDNN Install
https://developer.nvidia.com/rdp/cudnn-download
cuDNN v5.1 Runtime Lirary for Ubuntu 14.04(Deb)
(5) TensorFlow Install
$ pip install tensorflow-gpu
$ pip install keras
$ pip install h5py

Best Practices for ML Engineering

◼️Rules of Machine Learning: Best Practices for ML Enginnering
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf

The first rule states "Do not be afraid to launch a product without machine learning". It is suggested that it is better to write the rules yourself, especially when there are few data available for machine learning.
Being able to understand and use machine learning is a necessary condition as a machine learning engineer, but be careful that it is not sufficient condition. The truly important thing is that you can select the optimal method including the heuristic rule base as well.

Character-level CNN

Morphological analysis is used to decompose sentences in Japanese into words. Tools for morphological analysis, such as MeCab, can be used to break sentences into words. However, this morpheme analysis has weaknesses. That means that it can not be used for unknown words.

So there is Character - level CNN. (https://arxiv.org/abs/1509.01626)
Character-level CNN is a method that began to be seen around 2015. One of the features is that no morphological analysis is necessary. Enter the sentences in CNN in character units, put them in the feature map, and pour them into the entire tie layer to learn the classification.

CNN and RNN in natural language processing

In deep learning, RNN (Recurrent Neural Network) is often used for natural language processing. RNN is good at handling sequential inputs. However, in general, RNN learns slower than CNN. Therefore, papers using CNN for natural language processing came out frequently from around last year. For example, recently, the topic that Facebook implements machine translation with CNN came out on 2017/5.

https://code.facebook.com/posts/1978007565818999/a-novel-approach-to-neural-machine-translation/

Cloud service for Deep Learning

Amazon EC2 P2 Instance
　・Choose a Virginia northern region
　・AMI(Amazon Machine Image)
Deep Learning AMI Ubuntu Version

Google Cloud Platform

Ruby on Rails ver 5.1.0 Release

http://weblog.rubyonrails.org/2017/4/27/Rails-5-1-final/

When 5.1 is released, bug fixes apply only to 5.1.x.
Normal security fixes apply to 5.1.x and 5.0.x.
Serious security fixes apply to 5.1.x, 5.0.x and 4.2.x.
That is, 4.x is basically not supported.

Mastodon

Mastodon is an open source SNS developed by 24-year-old German Eugen Rochko. The biggest feature is that there is more than one server called Instance, not one service controlled centrally by Twitter or Facebook.

Each instance is one that can be said as a small Twitter, and it is completed as an SNS by itself. However, there is a mechanism to follow users of other instances called remote follow, basically any instance you register, you can interact with Mastodon users all over the world.

You can check the instance list in Mastodon Instances.

Ruby 2.4.1 release

Install rbenv

$ git clone https://github.com/rbenv/rbenv.git ~/.rbenv
$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc

$ echo 'eval "$(rbenv init -)"' >> ~/.bashrc
$ git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

Install ruby
$ rbenv install -l

$ rbenv global 2.4.1

Responding to Symantec's SSL certificate by Google

According to the contents of the following mailing list,
https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/eUAKwjihhBs/rpxMXjZHCQAJ

Since January 19, the Google Chrome team has been investigating a series of failures by Symantec Corporation to properly validate certificates. Over the course of this investigation, the explanations provided by Symantec have revealed a continually increasing scope of misissuance with each set of questions from members of the Google Chrome team; an initial set of reportedly 127 certificates has expanded to include at least 30,000 certificates, issued over a period spanning several years. This is also coupled with a series of failures following the previous set of misissued certificates from Symantec, causing us to no longer have confidence in the certificate issuance policies and practices of Symantec over the past several years. To restore confidence and security of our users, we propose the following steps:

• A reduction in the accepted validity period of newly issued Symantec-issued certificates to nine months or less, in order to minimize any impact to Google Chrome users from any further misissuances that may arise.

• An incremental distrust, spanning a series of Google Chrome releases, of all currently-trusted Symantec-issued certificates, requiring they be revalidated and replaced.

• Removal of recognition of the Extended Validation status of Symantec issued certificates, until such a time as the community can be assured in the policies and practices of Symantec, but no sooner than one year.

The SSL certificate of Symantec is the SSL certificate issued by the following organization.

$ wget -O - https://chromium.googlesource.com/chromium/src/+archive/master/net/data/ssl/symantec/roots.tar.gz | tar -x -z -f - -C roots/
$ for cert in roots/*.pem; do openssl x509 -inform pem -in $cert -noout -text | grep Issuer; done | grep -o 'O=[^,]\+' | sort -u

O=Equifax
O=Equifax Secure
O=Equifax Secure Inc.
O=GeoTrust Inc.
O=RSA Data Security
O=Symantec Corporation
O=TC TrustCenter for Security in Data Networks GmbH
O=TC TrustCenter GmbH
O=thawte
O=Thawte
O=Thawte Consulting
O=Thawte Consulting cc
O=The USERTRUST Network
O=VeriSign

Distrusting WoSign and StartCom Certificates

I have a website that uses a StartCom Class 1 Root DV Certificate for encryption & security.
Now, whenever I try to go to my website using the encrypted protocol, I get a privacy error. The exact error is: ERR::CERT_COMMON_NAME_INVALID
When I checked on my StartCom dashboard, it said that all StartCom certificates had been untrusted by Chrome, Mozilla, and Safari.
The reason was written in the following URL.

https://security.googleblog.com/2016/10/distrusting-wosign-and-startcom.html

https://docs.google.com/document/d/1C6BlmbeQfn4a9zydVi2UvjBGv6szuSB4sMYUcVrR8vQ/preview#

https://blog.mozilla.org/security/2016/10/24/distrusting-new-wosign-and-startcom-certificates/

https://support.apple.com/en-us/HT202858

Approach to increase productivity

There are two approaches to increase productivity, but the order of tackling is crucial.

First, reduce things to do.
Next, devise a way.

To increase productivity is neither tracing a heavy load nor sending a life of life.
It is the purpose of raising productivity to have room to have new opportunities at any time.

Hacker mind

Originally derived from values ​​commonly held by programmers active in MIT and around the 1960s and 1970s. They seriously thought that computers could improve society or create art. We deny the past without regard to the established concept, pursue technology as curious as we go, and create different values ​​until now. That is hacker mind.

In reality, in the world, things are born before the pure feelings of programmers' individuals' wanting to make something like this' or 'Would not it be interesting if there is such a thing'? Innovation happens as a result, not as aimed at causing innovation. People who can produce things and services like that are hackers.

picoCTF (after that)

I continued picoCTF, but the score was 4365 points, the ranking improved.
I ranked second in my company. Also, within the company group, I am in the 8th place. Yay!

picoCTF

picoCTF is a computer security game targeted at middle and high school students. The game consists of a series of challenges centered around a unique storyline where participants must reverse engineer, break, hack, decrypt, or do whatever it takes to solve the challenge. The challenges are all set up with the intent of being hacked, making it an excellent, legal way to get hands-on experience.

I also tried challenging. There were also quite a difficult problem, it was about a little over 1000 points. I think I will do my best to study security technology a little more.

Deep learning for natural language processing

Deep learning has attracted a great deal of attention as a result of image recognition and speech recognition.

In the field of natural language processing, utilization of deep learning is delayed, but one of the causes is presumed to be the difference in quality of input data.

Input data for image recognition and speech recognition consists of continuous values, but the input data of natural language processing is largely different in that it consists of discrete values. In the case of Japanese, it is composed of letters such as "a" and "i", and the closeness between letters is not defined.
Therefore, the tasks are as follows.

· How can I input sentences of arbitrary length?
· How to express words, phrases, sentences?

The keywords to solve the above problem are as follows.

- Vector Space Model, VSM
- word2vec
- Recursive Autoencoder
- Recursive Neural Network

HTTP/2

In the position of HTTP/2 being the evolutionary version of this SPDY, specifications were advanced by IETF (Internet Engineering Task Force) in May 2015 to achieve RFC conversion.

HTTP/2 has no effect unless both server side and client side support HTTP/2. However, even if the client side is incompatible with HTTP/2, communication can not be disabled, but communication can be performed using conventional HTTP/1.1 .

Whether the web site is compatible with HTTP/2 can be easily checked by using the browser plugin.

"HTTP/2 and SPDY indicator" plug-in

My site (https://hhoshina.info) has already corresponded to HTTP/2.

WordPress 4.7.2 Security Release

WordPress versions 4.7.1 and earlier are affected by three security issues:

4. An unauthenticated privilege escalation vulnerability was discovered in a REST API endpoint. Reported by Marc-Alexandre Montpas of Sucuri Security. *

https://wordpress.org/news/2017/01/wordpress-4-7-2-security-release/

DeepLearning: Learning to Learn(meta-learning)

The trend of DeepLearing is written in the following article.
●10 Deep Learning Trends and Predictions for 2017
●Deep Learning: The Unreasonable Effectiveness of Randomness

I am paying attention to "meta-leraning" in the article.

----Below, excerpts of articles----
The first is the mind boggling discovery that you can train a neural network to learn to learn (i.e. meta-learning). More specifically, several research groups have trained neural networks to perform stochastic gradient descent (SGD). Not only have they been able to demonstrate neural networks that have learned SGD, the networks have performed better than any hand tuned human method! The two papers that were submitted were”Deep Reinforcement Learning for Accelerating the Convergence Rate” and “Optimization as a Model for Few-Shot Learning” . Unfortunately though, these two groups have been previously scooped by Deep Mind, who showed that you could do this in this paper “Learning to Learn by gradient descent by gradient descent“. The two latter papers trained an LSTM, while the first one trained via RL. I had thought that it would take a bit longer to implement meta-learning, but it has arrived much sooner than I had expected!

Demand for Java Programmers ?

The position of the Java programmer can be categorized as "client side" "server side" "smart phone application side".

●Java region on client side

In many cases, end users are accustomed to Windows when their company or client seeks client side applications, so many people are requesting Windows Native applications developed with VisualBasic or C #.
Therefore, it can be said that there are many cases where the stage of the activity of Java programmers on the client side is not large.

●Java region on the server side

The Windows system is major to large, while the UNIX system is often used in a major to super big system. In both cases there are many requirements to develop in the Java language, and Java programmers are always required. It can be said that it is almost divided in Java or .NET framework.

The server side application is based on the Java servlet, but by using the framework such as Struts, Java EE, Spring Framework, Play Framework, it is required to simplify implementation of complicated functions, A Java programmer who understands the framework is required.

●Java region of the smartphone application side

Today, there are a great many demands for engineers who can develop application programs for Android.
In Japan, iOS users are more than Android, but Android is more worldwide.

As a business solution, it is not necessary to be multi-OS compliant, and sometimes it is compatible only with Android. This is because if you can limit the end user's terminal to Android, the iOS version becomes unnecessary. For example, it is fully conceivable to limit Android to terminals for insurance company agencies and sales outlets. Even if it is multi-OS compatible, at least the basic design is standardized, after that, it is conceivable that another team develops iOS version and Android version applications.

●Market for which Java programmers will be required in the future

Recently, it is rare to use Java for small projects. For smaller projects, more productive items like PHP and Ruby on Rails are used.
Java programmers continue to be required to be active in the medium-sized segment.

In the case of a medium-scale system or larger, expansion according to the business situation is indispensable. Once developed in the Java language system, Java programmers are responsible for renovation.

DeepLearning : Caffe Install

Image Analysis The procedure for installing "Caffe" of AI software is as follows.

*** OS
Ubuntu 14.04

**Caffe URL
http://caffe.berkeleyvision.org/

*** General dependencies
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
$ sudo apt-get install --no-install-recommends libboost-all-dev
$ sudo apt-get install libatlas-base-dev
$ sudo apt-get install python-dev
$ sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

*** caffe install
$ cd /usr/local
$ git clone https://github.com/BVLC/caffe
$ cd caffe
$ cp Makefile.config.example Makefile.config
$ vim Makefile.config
---
CPU_ONLY := 1
---
$ make clean
$ make all
$ make test
$ make runtest
$ sudo apt-get install libhdf5-serial-dev
$ sudo apt-get install libhdf5-mpi-dev
$ sudo apt-get install libhdf5-openmpi-dev
$ sudo apt-get install mpi-default-dev
$ make pycaffe
$ make distribute

***　numpy インストール
$ sudo apt-get install python-numpy
$ sudo pip install pandas
$ sudo apt-get install libjpeg8 libjpeg62 libjpeg62-dev libfreetype6 libfreetype6-dev zlib1g-dev
$ sudo pip install Pillow
$ sudo apt-get install python-scipy
$ sudo pip install scikit-image
$ sudo apt-get install python-protobuf

***　flask install
$ sudo pip install Flask
$ sudo apt-get install libxslt-dev
$ sudo pip install tornado
$ sudo apt-get install libyaml-dev
$ sudo pip install PyYAML

*********
$ vim .bashrc
—add---
export PYTHONPATH="/usr/local/caffe/python:${PYTHONPATH}"
—add---

***Test
$ cd /usr/local/caffe/data/ilsvrc12
$ ./get_ilsvrc_ux.sh
$ cd /usr/local/caffe/scripts/
$ ./dowload_model_binary.py ../models/bvlc_reference_caffenet
$ cd /usr/local/caffe/exapmles/web_demo/
$ python app.py -p 3000

SSL Setting

I checked the SSL setting of my server. The result was "A +". I will enumerate points devised by setting.

/etc/apache2/sites-enabled/default-ssl.conf

Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"

SSLProtocol ALL -TLSv1 -SSLv2 -SSLv3

SSLHonorCipherOrder on
SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5:!RC4:!DES:!DH:

Taking security check of Nessus

I conducted a vulnerability check of servers managed by Nessus.
As a result, a vulnerability was found in the configuration of the SSH server. The contents are as follows.

Description
Nessus has detected that the remote SSH server is configured to use the Arcfour stream cipheror no cipher at all. RFC 4253 advises against using Arcfour due to an issue with weak keys.
Nessus has detected that the remote SSH server is configured to use the Arcfour stream cipheror no cipher at all. RFC 4253 advises against using Arcfour due to an issue with weak keys.

Solution

Contact the vendor or consult product documentation to remove the weak ciphers.

See Also

https://tools.ietf.org/html/rfc4253#section-6.3

I made the following change of SSH service setting.

$ ssh -Q mac | awk 'BEGIN{printf "MACs "};(!/cbc/ && !/md/ && !/sha1/ && !/rc4/ && !/arcfour/){printf "%s,",$0};END{printf "\n"}' | sed -e 's/,$//g'
MACs hmac-sha2-256,hmac-sha2-512,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com

$ ssh -Q cipher | awk 'BEGIN{printf "Ciphers "};(!/cbc/&&!/rc4/&&!/arcfour/){printf "%s,",$0};END{printf "\n"}' | sed -e 's/,$//g'
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com

The character string output by the command is reflected in the following file.

/etc/ssh/sshd_config
/etc/ssh/ssh_config

$ sudo service sshd restart

After upgrading to iOS 10.2.1….

After upgrading to iOS 10.2.1 iPhone owners are reporting the update has broken Apple’s fingerprint reader.

The problem primarily appears to impact iPhone 6 and iPhone 6 Plus owners and logical options such as removing and re-adding fingerprints and a factory reset doesn’t work. Interestingly downgrading to iOS 10.2 is reported to fix the problem, which isolates iOS 10.2.1 as the root cause.

apache2.4.23 -> 2.4.25

In the case of Ubuntu 16.04 LTS, the version of apache 2 is old. It is apache 2.4.23, but since there is also a security hole (CVE-2016-4979), change to the latest version of apache2.
The procedure is as follows.

$ apache2 -v
Server version: Apache/2.4.23 (Ubuntu)
Server built: 2016-07-11T00:00:00

$ sudo add-apt-repository ppa:ondrej/apache2
$ sudo apt-get update
$ sudo apt-get upgrade

$ apache2 -v
Server version: Apache/2.4.25 (Ubuntu)
Server built: 2016-12-21T09:13:44

Progressive Web Apps

Native applications are certainly easy to use and engagement with users is high. However, it has to be developed for iOS and Android respectively, and maintenance cost is high.

For the above problem, it is possible to develop hybrid applications that can be used on multiple platforms such as Titanium, Xamarin, PhoneGap. However, it only reduces the cost of developing for each platform, and does not make deployment easier.

Proposed there is Progressive Web Apps rather than Hybrid. Rather than creating a hybrid application that runs on each platform, making applications that run on the Web can realize a movement closer to native applications.

Also, Google has launched "WebAPK" that enables you to use WEB applications like ordinary Android applications for Android Chrome Dev and Chrome Canary.

Learning Deep Learning Framework

I was writing in the past, programming, full scratch. For example, I wrote a CGI program in Python. However, as for the Web, I started using Ruby on Rails of the web application Framework.

Now, in order to study DeepLearning, I tried to make a simple AI program with scratch, but judged that it is more efficient to use DeepLearning's framework such as TensorFlow.

Twine

I bought Twine five years ago.
Recently I received a message from Supermechanical Inc. that I have released a new product.
Unfortunately, Twine is already out of sale.