Information theory and coding

Description: The discipline is one of the main profiling disciplines and includes the study of the following issues: The concept of information, entropy. Information compression. Discrete channels and their properties. The speed of information transfer in the channel. The bandwidth of the channel. Shannon's direct coding theorem for a channel without memory. Inversion of the Shannon coding theorem. Noise-resistant coding.

Amount of credits: 6

Пререквизиты:

• Of Informatively-communication technologies

Course Workload:

Component: Component by selection

Cycle: Profiling disciplines

Goal

• The course studies the basic models of discrete sources of information and discrete channels, defines the concept of entropy, considers the basic theorems for discrete sources and channels, studies the issues of information compression, and considers the main noise-immune codes

Objective

• familiarization of students with the main processes occurring during converting messages into a signal and their transmission over channels and communication lines
• mastering by students of general issues of building collection systems, transmission and processing of information

Learning outcome: knowledge and understanding

• the student must know and understand how information is measured, the laws of changing the amount of information during its transformation, what means exist to combat interference, how information compression algorithms work
• the main phases and principles of its application in the development of computer technology and software

Learning outcome: applying knowledge and understanding

• the student should be able to navigate the effectiveness of the chosen coding method
• apply the main models and means of information transfer to optimization of modern computer systems

Learning outcome: formation of judgments

• the student should talk about knowledge about the properties of entropy, know the definitions of an ergodic source, a channel, be able to prove the basic coding theorems for discrete sources and channels, know the structure of the main error-correcting codes, know the limit compression estimates information

Learning outcome: communicative abilities

• development and improvement of communicative abilities of students
• development of skills to participate in a constructive dialogue about the role and importance of artificial intelligence systems in the modern world, various areas in artificial intelligence systems

Learning outcome: learning skills or learning abilities

• formation of skills in the field of artificial intelligence systems for the implementation of information coding tasks
• developing skills in applying information coding methods
• the ability to contribute, within academic and professional contexts, to technological, social or cultural development in the interest of building a knowledge society

Teaching methods

lectures and online lectures, laboratory classes using slides and other multimedia tools.

Assessment of the student's knowledge

Teacher oversees various tasks related to ongoing assessment and determines students' current performance twice during each academic period. Ratings 1 and 2 are formulated based on the outcomes of this ongoing assessment. The student's learning achievements are assessed using a 100-point scale, and the final grades P1 and P2 are calculated as the average of their ongoing performance evaluations. The teacher evaluates the student's work throughout the academic period in alignment with the assignment submission schedule for the discipline. The assessment system may incorporate a mix of written and oral, group and individual formats.

The evaluating policy of learning outcomes by work type

Evaluation form

The student's final grade in the course is calculated on a 100 point grading scale, it includes:

• 40% of the examination result;
• 60% of current control result.

The final grade is calculated by the formula:

 

Where Midterm 1, Midterm 2are digital equivalents of the grades of Midterm 1 and 2;

E is a digital equivalent of the exam grade.

Final alphabetical grade and its equivalent in points:

The letter grading system for students' academic achievements, corresponding to the numerical equivalent on a four-point scale:

Topics of lectures

• Понятие информации, энтропии
• Системы связи
• Взаимная информация и её свойства
• Задача кодирования дискретного источника кодами равной длины
• Понятие скорости кодирования
• Прямая и обратная теоремы кодирования Шеннона дискретного источника кодами равной длины
• Задача кодирования дискретного источника кодами неравной длины
• Разрешимость задачи определения однозначной дешифрируемости
• Алгоритмы построения оптимальных кодов (Фано, Шеннона, Хаффмена)
• Словарные методы сжатия информации
• Дискретные каналы и их свойства
• Скорость передачи информации в канале
• Прямая теорема кодирования Шеннона
• Теория помехоустойчивого кодирования
• Граница Хэмминга

Key reading

• Белов, В.М. Теория информации. Курс лекций: Учебное пособие / В.М. Белов, С.Н. Новиков, О.И. Солонская. - М.: ГЛТ, 2019. - 143 c.
• Малюк, А.А Теория защиты информации / А.А Малюк. - М.: ГЛТ, 2018. - 184 c.
• Осокин, А.Н. Теория информации: Учебное пособие для прикладного бакалавриата / А.Н. Осокин, А.Н. Мальчуков. - Люберцы: Юрайт, 2021. - 205 c.

Further reading

• Ворожцов, А.В. Путь в современную информатику: Комбинаторика, анализ, теория графов, теория игр, моделированию, теория информации, логика и теория множеств / А.В. Ворожцов. - М.: Ленанд, 2020. - 144 c.
• Петров, В.М. Искусствознание и теория информации: Сборник научных статей / В.М. Петров, А.В. Харуто. - М.: Красанд, 2019. - 432 c.